CN108053435A - Dynamic realtime three-dimensional rebuilding method and system based on handheld mobile device - Google Patents
Dynamic realtime three-dimensional rebuilding method and system based on handheld mobile device Download PDFInfo
- Publication number
- CN108053435A CN108053435A CN201711228561.7A CN201711228561A CN108053435A CN 108053435 A CN108053435 A CN 108053435A CN 201711228561 A CN201711228561 A CN 201711228561A CN 108053435 A CN108053435 A CN 108053435A
- Authority
- CN
- China
- Prior art keywords
- mobile device
- handheld mobile
- dimensional reconstruction
- dynamic
- depth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Abstract
The present invention provides a kind of dynamic realtime three-dimensional rebuilding method and system, method based on handheld mobile device and includes the following steps：The coloured image and depth image of synchronous acquisition scene；Lossless compression is carried out to coloured image and depth image data；Data synchronous transfer is to processing unit after compressing；Decompression obtains color image sequence and range image sequence；According to the transition matrix between the color camera and depth camera demarcated；Carry out dynamic three-dimensional reconstruction；The model of dynamic three-dimensional reconstruction is subjected to texture mapping, the model after texture mapping is transferred to handheld mobile device and is shown on handheld mobile device.The present invention is more convenient to the stationary body in scene and the dynamic three-dimensional reconstruction process of moving object, and moving object in scene and scene is allow to obtain the result of three-dimensional reconstruction；By dynamic realtime three-dimensional reconstruction algorithm, accurate Three-dimension Reconstruction Model can be obtained, including better model geometric details, reflectivity and action message.
Description
Technical field
The present invention relates to the three-dimensional reconstruction field in computer vision more particularly to a kind of moving based on handheld mobile device
State real-time three-dimensional method for reconstructing and system.
Background technology
With the continuous development of the three-dimensional reconstruction in computer vision field, the three-dimensional reconstruction of dynamic scene into
For a research hotspot in computer realm, the three-dimensional rebuilding method of new dynamic scene is constantly emerged in large numbers, reconstructed
Precision and geometric detail of model and other effects are also constantly being promoted.
In terms of algorithm, current trend progress dynamic scene three-dimensional rebuilding method using more scenes input,
Control ambient enviroment goes to recover geometry, surface and the movable information of a scene using initialization model or action priori etc..Perhaps
More synchronous algorithms for rebuilding dynamic scene all play an important role in computer vision and graphics field.For example, rebuild geometry and
The model of surface information can be used in the video of the free view-point of virtual reality technology.Although the three-dimensional reconstruction skill of dynamic scene
Art has had a great development, but due to excessively huge solution space, the three-dimensional reconstruction of dynamic scene is caused to take the photograph fine
As environment, picture pick-up device and processing equipment have certain requirement.
It is rebuild in the colour and depth camera of single-view in the technology of dynamic scene, it is preferably dynamic effect has occurred
State real-time reconstruction algorithm, however existing algorithm use it is mostly be the tripod constant depth video camera that can not be interacted, such as Kinect
Depth camera etc..The dynamic realtime based on handheld mobile device for observing result in phase generator terminal by graphic user interface is three-dimensional
Rebuild practicable method not yet.
The content of the invention
The present invention in order to solve in the prior art not over graphic user interface phase generator terminal observation result based on hand
It is three-dimensional to provide a kind of dynamic realtime based on handheld mobile device for the problem of holding the dynamic realtime three-dimensional rebuilding method of mobile equipment
Method for reconstructing and system.
To solve the above-mentioned problems, the technical solution adopted by the present invention is as described below：
A kind of method of the dynamic realtime three-dimensional reconstruction based on handheld mobile device is provided, is included the following steps：S1：It utilizes
The colour of the color camera of handheld mobile device and the depth camera synchronous acquisition scene that can be fixed on handheld mobile device
Image and depth image, handheld mobile device are mobile phone, tablet or video camera；S2：By the coloured image and depth image of acquisition
Data carry out lossless compression as straight binary data, and the method that lossless compression uses is LZ4 compression algorithms；S3：Using
Socket communications pass compressed binary color image data with depth image binary data by the way that two passes are synchronous
Processing unit is defeated by, processing unit is desktop computer or high in the clouds；S4：To binary color image data and depth image binary number
Color image sequence and range image sequence are obtained according to decompression；S5：According between the color camera and depth camera demarcated
Transition matrix, the range image sequence range image sequence that decompression obtains being converted under the three-dimensional system of coordinate of color camera；
S6：It is carried out according to the color image sequence that transformed range image sequence and decompression obtain based on single-view color depth camera
Geometry, reflectivity and the dynamic three-dimensional reconstruction of action；Include the following steps：S61：Joint motions and illumination optimization：From current
Non-rigid surface's movement and low frequency environments illumination are estimated in photographed frame and the obtained static models of previous frame；S62：Static models
Update：New movement is solved according to the coloured image of current shooting frame and depth image, is solved instead further according to low frequency environments illumination
Rate is penetrated, so as to carry out the update of static models.S7：The model of dynamic three-dimensional reconstruction is subjected to texture mapping, transmission uses
Model after texture mapping is transferred to handheld mobile device and is shown on handheld mobile device by Socket communications, and model passes through
Application software is shown on handheld mobile device, and model is adjusted by application software.
A kind of system of the dynamic realtime three-dimensional reconstruction based on handheld mobile device is provided, including：With color camera
Handheld mobile device for obtaining the coloured image of scene, and shows, adjusts Three-dimension Reconstruction Model；Depth camera is fixed on, is used
In the depth image for synchronously obtaining scene；The processing unit that can carry out data transmission with handheld mobile device and depth camera,
For handling coloured image and depth image.In alternative embodiments, further include memory, processor and be stored in described deposit
In reservoir and the computer program that can run on the processor, the processor are realized such as when performing the computer program
The step of claim 1-8 any the methods.
Beneficial effects of the present invention are：There is provided a kind of method of the dynamic realtime three-dimensional reconstruction based on handheld mobile device and
System combines synchronous acquisition coloured image and depth image with depth camera by handheld mobile device, can make in scene
Stationary body and the dynamic three-dimensional reconstruction process of moving object become more convenient, are moved moving object in scene and scene
The result of state three-dimensional reconstruction；By dynamic realtime three-dimensional reconstruction algorithm, accurate Three-dimension Reconstruction Model can be obtained, including more preferable
Model geometric details, reflectivity and action message.
Description of the drawings
Fig. 1 is the method schematic diagram of the dynamic realtime three-dimensional reconstruction based on handheld mobile device in the embodiment of the present invention.
Fig. 2 is the method schematic diagram of dynamic realtime three-dimensional reconstruction in the embodiment of the present invention.
Specific embodiment
The present invention is described in detail by specific embodiment below in conjunction with the accompanying drawings, for a better understanding of this hair
It is bright, but following embodiments are not intended to limit the scope of the invention.In addition, it is necessary to illustrate, the diagram provided in following embodiments
Only illustrate the basic conception of the present invention in a schematic way, the component related with the present invention is only shown in attached drawing rather than according to reality
Component count, shape during implementation and size are drawn, during actual implementation shape, quantity and the ratio of each component can be it is a kind of with
The change of meaning, and its assembly layout form may also be increasingly complex.
Embodiment 1
Three-dimensional reconstruction
Three-dimensional reconstruction refers to establish three-dimensional body the mathematical model for being suitble to computer representation and processing, is in computer ring
The basis of its property is handled it, operated and analyzed under border, and establishes the virtual of expression objective world in a computer
The key technology of reality.In computer vision, three-dimensional reconstruction refers to three-dimensional according to the image reconstruction of single-view or multi views
The process of information is due to the INFORMATION OF INCOMPLETE of single video, therefore three-dimensional reconstruction is needed using Heuristics and the three-dimensional of multi views
Rebuild (binocular of similar people positions) relatively easily, method is that first video camera is demarcated, that is, calculates video camera
Image coordinate system and world coordinate system relation then using the information reconstruction in multiple two-dimensional images go out three-dimensional information.
The general step of three-dimensional reconstruction
(1) image acquisition：Before image procossing is carried out, the two dimensional image of three-dimensional body is first obtained with video camera.Light
Very big influence is caused to subsequent image procossing according to condition, geometrical property of camera etc..
(2) camera calibration：Effective imaging model is established by camera calibration, solves the inside and outside ginseng of video camera
Number, the matching result that can thus combine image obtain the three-dimensional point coordinate in space, and three-dimensional reconstruction is carried out so as to reach
Purpose.
(3) feature extraction：Feature mainly includes characteristic point, characteristic curve and region.All it is in most cases with characteristic point
For Matching unit, characteristic point is extracted in what manner with being closely connected with which kind of matching strategy.Therefore carrying for characteristic point is being carried out
Which kind of need first to determine with matching process when taking.
Feature point extraction algorithm can be summarized as：Method based on directional derivative, the side based on image intensity contrast's relation
Method, three kinds of the method based on mathematical morphology.
(4) Stereo matching：Stereo matching refers to establish a kind of corresponding pass between image pair according to the feature extracted
System, that is, imaging point of the same physical space o'clock in two width different images correspond.It is being matched
When it is noted that in scene some factors interference, such as illumination condition, noise jamming, scene geometry distortion, surface physics
Many changing factors such as characteristic and video camera machine characteristic.
(5) three-dimensional reconstruction：There is more accurate matching result, with reference to the inside and outside parameter of camera calibration, it is possible to extensive
It appears again three-dimensional scene information.Since reconstruction accuracy is by the influence of the factors such as matching precision, the inside and outside parameter error of video camera,
Therefore firstly the need of the work for carrying out the several steps in front so that the precision of links is high, and error is small, so can just design
One more accurate stereo visual system.
The method of three-dimensional reconstruction
The video camera that traditional stereo imaging system is put together using two, it is parallel to watch object to be reconstructed attentively.It is similar
The mankind are stacked by the image that eyes perceive calculates that (actually human brain permits the perception course complexity of depth information to depth certainly
It is more), if known two video cameras to each other away from focal length, and intercept two pictures of left and right can successfully overlap, then
Depth information can push away rapidly.This method has to rely on effective picture pixels the matching analysis, is generally compared using block or right
Pole geometric algorithm is reached.
Binocular vision method also known as is done using the stereo vision method of two video cameras, separately there are three visions to be used with other more
The extension method of video camera.
It is solved in the colorimetry model being pre-designed using the brightness value substitution of image pixel, solution, that is, depth letter of equation
Breath.Since the unknown number in equation group crosses restrictive condition more, the scope of disaggregation must be reduced by more assumed conditions.Such as
Add in surface differentiable property (differentiability), curvature limitation (curvatureconstraint), smooth degree
(smoothness) and more limit to acquire accurate solution.Stereoptics method is derived after this method.
In order to make up the information of single photo offer in luminosity forming process deficiency, stereoptics method is shot using a camera
Multiple pictures, the shooting angle of these photos is identical, and difference therein is the lighting condition of light.Simplest cubic light
Method uses three light sources, and determinand is irradiated in the direction different from three, only opens a light source every time.It is comprehensive again after the completion of shooting
It closes three photos and the gradient vector of body surface is solved using perfect diffusion (perfectdiffusion) model in optics
(gradients), it can obtain threedimensional model after the integration of vector field.This method is not particularly suited for smooth bright without being similar to
The object of primary surface (Lambertiansurface).
Such method is to form three-dimensional body using a series of lines of outline of objects.When the part surface of object can not be
Profile is online when showing, and three-dimensional information will be lost after reconstruction.Common mode is that determinand is positioned on electric turntable, every time
Its image is shot after rotating a low-angle, then removes background via image procossing skill and takes out lines of outline, collects each angle
Contour line after " can delineate " into threedimensional model.
In addition some methods need user to provide information in reconstruction process, by the special performance of human visual system,
Auxiliary completes reconstruction algorithm.
These modes are all based on photo physics of photography, for same a object captured image to calculate three-dimensional information.It is another
Kind similar mode is that panorama is rebuild, and is to be allowed to be able to rebuild scene environment in fixed point photographs circumferential images.
Common video camera can provide abundant color and texture information, this is highly useful for three-dimensional reconstruction, but
It is the influence that its data captured is highly prone to illumination variation, while in some complex scenes, the texture of background and prospect
It may be highly similar.Compared with traditional two dimensional gray or coloured image, depth image can provide reference object to camera
Range information.Using the depth value of each location of pixels and camera internal parameter information, can easily to reference object into
Row shape and dimensioning, meanwhile, because the influence that depth data receives surface texture is less, Target Segmentation, background shearing
The problems such as become to be more prone to.
There are mainly three types of forms for depth camera at present：Depth camera based on binocular vision, the depth phase based on structure light
Machine and the depth camera based on TOF (time flight method).It is briefly described below, no matter which kind of form may be used in
In embodiment.
Depth camera based on binocular vision is using binocular vision technology, utilizes two cameras pair for being in different visual angles
The same space is taken pictures, the difference of pixel where same object and the depth where the object in the image that two cameras are shot
It spends directly related, thus depth information is obtained by calculating pixel deviations by image processing techniques.
Based on the depth camera of structure light by gathering mesh to object space projection coding structure light pattern, then by camera
The image of structured light patterns is contained in mark space, then handles the image and such as carries out matching meter with reference configuration light image
Calculation etc. can directly obtain depth information.
Based on the depth camera of TOF by emitting laser pulse to object space, laser pulse is connect after target reflects
It receives after unit receives and records the turnaround time of laser pulse, the depth information of target is calculated by the time.These three
The generally acquisition color camera of the first in method, thus it is big by illumination effect, while the calculation amount for obtaining depth information is larger.Afterwards
Two kinds generally utilize infrared light, and from illumination effect, while calculation amount is relatively small.Current double cameras utilize VCSEL chips
Structure light, TOF camera as light source can be embedded into the equipment such as mechanical, electrical brain in one's hands.
Microsoft has issued Kinect imaging devices, and the equipment is by depth transducer (being based on structure light principle) and RGB
Video camera two parts form.The picking rate of the equipment can reach 30 frames/second, resolution ratio 640*480, depth data
Effective range is about that the depth data accuracy rate between 0.5-10 meters, wherein 0.8-3.5 meters is higher.Other RGBD cameras, than
Kinect Xbox 360, Kinect One, Xtion and Orbbec such as also occurs therewith.
In order to solve in the prior art not over graphic user interface phase generator terminal observation result based on hand-held depth
The problem of dynamic realtime three-dimensional rebuilding method of video camera, provides a kind of dynamic realtime Three-dimensional Gravity based on hand-held depth camera
Construction method and system.
As shown in Figure 1, a kind of method of the dynamic realtime three-dimensional reconstruction based on handheld mobile device is provided, including walking as follows
Suddenly：
(1) it is synchronous with the depth camera that can be fixed on handheld mobile device using the color camera of handheld mobile device
The coloured image and depth image of scene are gathered, handheld mobile device is mobile phone, tablet or video camera, can also include existing skill
Other handheld devices in art, as long as depth camera can be installed；
(2) coloured image of acquisition and depth image data are subjected to lossless compression as straight binary data, it is lossless
The method that compression uses is LZ4 compression algorithms, can also use the compression algorithms such as ZLIB, SNAPPY；
(3) compressed binary color image data and depth image binary data are passed through using Socket communications
For two passes synchronous transfer to processing unit, processing unit is desktop computer or high in the clouds, because at desktop computer or the processor in high in the clouds
Reason ability is stronger, can handle larger data volume, can if the processor of similary processing capacity is mounted on mobile equipment
Make mobile weight of equipment, volume increase, it has not been convenient to use；High transfer rate router may be employed when being transferred to processing unit
The modes such as wifi can be used, as long as meeting quick transmission data；
(4) processing unit decompresses to obtain coloured image sequence to binary color image data and depth image binary data
Row and range image sequence；
(5) according to the transition matrix between the color camera and depth camera demarcated, the depth image that decompression is obtained
Sequence is converted into the range image sequence under the three-dimensional system of coordinate of color camera；The calibration of color camera and depth camera is using existing
There is the scaling method in technology, such as using Zhang Zhengyou standardizations.
(6) color image sequence obtained according to transformed range image sequence and decompression carries out colored based on single-view
Geometry, reflectivity and the dynamic three-dimensional reconstruction of action of depth camera；
As shown in Fig. 2, dynamic three-dimensional reconstruction includes the following steps：
Joint motions and illumination optimization：Estimate non-rigid table from current shooting frame and the obtained static models of previous frame
Face moves and low frequency environments illumination；
Static models update：New movement is solved according to the coloured image of current shooting frame and depth image, further according to low
Frequency ambient lighting solves reflectivity, so as to carry out the update of static models.
(7) model of dynamic three-dimensional reconstruction is subjected to texture mapping, transmission will be after texture mapping using Socket communications
Model is transferred to handheld mobile device and is shown on handheld mobile device, and model is by application software on handheld mobile device
It has been shown that, is adjusted model by application software.
In the alternative embodiments of the present embodiment, method when handheld camera carries out dynamic realtime three-dimensional reconstruction specifically includes
Following steps：
(1) using iPad Pro color camera and can be fixed on iPad Pro on and can give iPad Pro transmission number
According to depth camera structure sensor with the frame per second of 30 frames/second, the coloured image and depth image of synchronous acquisition scene；
(2) using LZ4 compression algorithms, using the coloured image of acquisition and depth image data as straight binary data into
Row lossless compression；
(3) using Socket communications by compressed binary data, by the router with high transfer rate from
IPad Pro are transferred to desktop computer, and establishing two threads, to transfer compressed coloured image and depth image respectively desk-top to ensure
Machine obtains the synchronism of depth image and colo r image transmission；
(4) data for passing iPad Pro back decompress on desktop computer, obtain 30 frames/second color image sequence and
Range image sequence；
(5) range image sequence that will be obtained after decompression, according to the conversion between the color camera and depth camera demarcated
Matrix is converted to the range image sequence under color camera three-dimensional system of coordinate；
(6) using transformed range image sequence and color image sequence as the dynamic realtime based on single RGBD cameras
The input of three-dimensional reconstruction algorithm carries out the three-dimensional reconstruction of dynamic realtime；
(7) model after reconstruction and the model picture after progress texture mapping are passed back iPad Pro using Socket communications
It is shown on the application interface of iPad Pro.
(8) repeat step (1) to step (7) and reach dynamic realtime three-dimensional reconstruction effect.
User using when specific embodiment comprise the following steps：
(1) router is connected on desktop computer, and ensure as server end desktop computer have it is corresponding with program
IP address.It is connected to using iPad Pro as client in the Small-scale LAN of desktop computer and router.
(2) iPad Pro with Mac portable computers are connected by data cable, the IOS programs set is downloaded simultaneously
On iPad Pro.After the completion of download, Structure Sensor are fixed on iPad Pro, and with data cable by two
Person is connected.
(3) desk-top generator terminal processing routine is first opened, program can wait the data from iPad Pro.It is then turned on iPad
The application program at Pro ends, by camera alignment target scene, one to three meter of distance is advisable.
(4) handheld camera is tried one's best remains stationary, and after waiting about 5 seconds after first frame model treatment, handling result can go out
On present iPad Pro Application Program Interfaces.Then target scene can be begun around to rotate to rebuild target dynamic scene.
The result of reconstruction can be illustrated in real time on iPad Pro Application Program Interfaces.
IPad Pro end programs and desk-top generator terminal program can repeat following steps by equipment：Utilize iPad Pro's
The coloured image and depth image of color camera and depth camera structure sensor synchronous acquisition scene；By the coloured image of acquisition
Lossless compression is carried out with depth image；Compressed data are transferred to desktop computer；The data passed back are carried out on desktop computer
Decompression, obtains color image sequence and range image sequence；The range image sequence obtained after decompression is converted into color camera
Range image sequence under three-dimensional system of coordinate；Dynamic realtime is carried out to transformed range image sequence and color image sequence
Three-dimensional reconstruction；Model after reconstruction and the model picture after progress texture mapping are passed back and including applying boundary in iPad Pro
On face.Repeat the step in block diagram 1.So far the dynamic realtime three-dimensional reconstruction based on hand-held depth camera is just realized.
The present invention also provides a kind of system of the dynamic realtime three-dimensional reconstruction based on handheld mobile device, including：With coloured silk
The handheld mobile device of form and aspect machine for obtaining the coloured image of scene, and shows, adjusts Three-dimension Reconstruction Model；It is fixed on depth
Camera is spent, for the synchronous depth image for obtaining scene；It can carry out data transmission with handheld mobile device and depth camera
Processing unit, for handling coloured image and depth image.
In the alternative embodiments of the present embodiment, further include memory, processor and be stored in the memory simultaneously
The computer program that can be run on the processor, the processor are realized when performing the computer program such as institute of the present invention
The step of stating method.
The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, it is impossible to assert
The specific implementation of the present invention is confined to these explanations.For those skilled in the art to which the present invention belongs, do not taking off
On the premise of from present inventive concept, several equivalent substitute or obvious modifications can also be made, and performance or purposes are identical, all should
When being considered as belonging to protection scope of the present invention.
Claims (10)
- A kind of 1. method of the dynamic realtime three-dimensional reconstruction based on handheld mobile device, which is characterized in that include the following steps：S1：Color camera and the depth camera synchronous acquisition that can be fixed on handheld mobile device using handheld mobile device The coloured image and depth image of scene；S2：The coloured image of acquisition and depth image are subjected to lossless compression as straight binary data；S3：Compressed binary color image data and depth image binary data are given by two passes synchronous transfer Processing unit；S4：Binary color image data and depth image binary data are decompressed to obtain color image sequence and depth image Sequence；S5：According to the transition matrix between the color camera and depth camera demarcated, the range image sequence that decompression is obtained The range image sequence being converted under the three-dimensional system of coordinate of color camera；S6：Dynamic three-dimensional reconstruction is carried out according to the color image sequence that transformed range image sequence and decompression obtain；S7：The model of dynamic three-dimensional reconstruction is subjected to texture mapping, the model after texture mapping is transferred to handheld mobile device simultaneously It is shown on handheld mobile device.
- 2. a kind of method of the dynamic realtime three-dimensional reconstruction based on handheld mobile device as described in claim 1, feature exist In handheld mobile device described in step S1 is mobile phone, tablet or video camera.
- 3. a kind of method of the dynamic realtime three-dimensional reconstruction based on handheld mobile device as described in claim 1, feature exist In the method that lossless compression described in step S2 uses is LZ4 compression algorithms.
- 4. a kind of method of the dynamic realtime three-dimensional reconstruction based on handheld mobile device as described in claim 1, feature exist In the transmission in step S3 and step S7 is communicated using Socket.
- 5. a kind of method of the dynamic realtime three-dimensional reconstruction based on handheld mobile device as described in claim 1, feature exist In processing unit described in step S3 is desktop computer or high in the clouds.
- 6. a kind of method of the dynamic realtime three-dimensional reconstruction based on handheld mobile device as described in claim 1, feature exist In the method for dynamic three-dimensional reconstruction is the dynamic of the geometry based on single-view color depth camera, reflectivity and action in step S6 State real-time three-dimensional is rebuild.
- 7. a kind of method of the dynamic realtime three-dimensional reconstruction based on handheld mobile device as claimed in claim 6, feature exist In dynamic realtime three-dimensional reconstruction includes the following steps：S61：Joint motions and illumination optimization：Estimate non-rigid table from current shooting frame and the obtained static models of previous frame Face moves and low frequency environments illumination；S62：Static models update：New movement is solved according to the coloured image of current shooting frame and depth image, further according to low Frequency ambient lighting solves reflectivity, so as to carry out the update of static models.
- 8. a kind of method of the dynamic realtime three-dimensional reconstruction based on handheld mobile device as described in claim 1, feature exist In the model in step S7 after texture mapping shows that the application software is to line by application software on handheld mobile device Model after reason textures is adjusted.
- 9. a kind of system of the dynamic realtime three-dimensional reconstruction based on handheld mobile device, which is characterized in that including：Handheld mobile device with color camera for obtaining the coloured image of scene, and shows, adjusts Three-dimensional Gravity modeling Type；Depth camera is fixed on, for the synchronous depth image for obtaining scene；The processing unit carried out data transmission with handheld mobile device and depth camera, for handling coloured image and depth map Picture.
- 10. the system of the dynamic realtime three-dimensional reconstruction of handheld mobile device as claimed in claim 9, which is characterized in that also wrap It includes memory, processor and is stored in the computer program that can be run in the memory and on the processor, it is described Processor realizes the step of the method as any such as claim 1-8 when performing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711228561.7A CN108053435A (en) | 2017-11-29 | 2017-11-29 | Dynamic realtime three-dimensional rebuilding method and system based on handheld mobile device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711228561.7A CN108053435A (en) | 2017-11-29 | 2017-11-29 | Dynamic realtime three-dimensional rebuilding method and system based on handheld mobile device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108053435A true CN108053435A (en) | 2018-05-18 |
Family
ID=62121342
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711228561.7A Pending CN108053435A (en) | 2017-11-29 | 2017-11-29 | Dynamic realtime three-dimensional rebuilding method and system based on handheld mobile device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108053435A (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108960225A (en) * | 2018-07-06 | 2018-12-07 | 四川奇迹云科技有限公司 | It is a kind of that auxiliary system is installed based on object identification and the dismantling of three-dimensional reconstruction |
| CN109872716A (en) * | 2018-12-25 | 2019-06-11 | 南京硅基智能科技有限公司 | A kind of plant cognition voice interactive system based on two dimensional code |
| CN109951695A (en) * | 2018-11-12 | 2019-06-28 | 北京航空航天大学 | Light field modulating three-dimensional imaging method and imaging system are moved freely based on mobile phone |
| CN109978931A (en) * | 2019-04-04 | 2019-07-05 | 北京悉见科技有限公司 | Method for reconstructing three-dimensional scene and equipment, storage medium |
| CN110349251A (en) * | 2019-06-28 | 2019-10-18 | 深圳数位传媒科技有限公司 | A kind of three-dimensional rebuilding method and device based on binocular camera |
| CN110634177A (en) * | 2018-06-21 | 2019-12-31 | 华为技术有限公司 | Object modeling movement method, device and equipment |
| WO2020038277A1 (en) * | 2018-08-21 | 2020-02-27 | 先临三维科技股份有限公司 | Image acquisition and processing methods and apparatuses for three-dimensional scanning, and three-dimensional scanning device |
| WO2020062919A1 (en) * | 2018-09-30 | 2020-04-02 | Oppo广东移动通信有限公司 | Data processing method, mec server and terminal device |
| WO2020192706A1 (en) * | 2019-03-25 | 2020-10-01 | 华为技术有限公司 | Object three-dimensional model reconstruction method and device |
| WO2021087810A1 (en) * | 2019-11-06 | 2021-05-14 | Oppo广东移动通信有限公司 | Information processing methods and systems, and encoding apparatus, decoding apparatus and storage medium |
| WO2021087800A1 (en) * | 2019-11-06 | 2021-05-14 | Oppo广东移动通信有限公司 | Information processing method, encoding apparatus, decoding apparatus, system, and storage medium |
| CN113870338A (en) * | 2020-06-30 | 2021-12-31 | 北京瓦特曼科技有限公司 | Three-dimensional reconstruction-based zinc bath slagging-off method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622762A (en) * | 2011-01-31 | 2012-08-01 | 微软公司 | Real-time camera tracking using depth maps |
| CN103106252A (en) * | 2013-01-16 | 2013-05-15 | 浙江大学 | Method for using handheld device to position plane area |
| CN105225256A (en) * | 2015-06-10 | 2016-01-06 | 西安交通大学 | A kind of display packing of high gray depth image |
| CN206400641U (en) * | 2017-01-13 | 2017-08-11 | 中国人民公安大学 | Portable scene of a crime rebuilds scanner and scene of a crime three-dimensional reconstruction system |
| CN107103626A (en) * | 2017-02-17 | 2017-08-29 | 杭州电子科技大学 | A kind of scene reconstruction method based on smart mobile phone |
| CN107256540A (en) * | 2017-06-12 | 2017-10-17 | 四川大学 | Image repair method, device and three-dimensional reconstruction system |
-
2017
- 2017-11-29 CN CN201711228561.7A patent/CN108053435A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622762A (en) * | 2011-01-31 | 2012-08-01 | 微软公司 | Real-time camera tracking using depth maps |
| CN103106252A (en) * | 2013-01-16 | 2013-05-15 | 浙江大学 | Method for using handheld device to position plane area |
| CN105225256A (en) * | 2015-06-10 | 2016-01-06 | 西安交通大学 | A kind of display packing of high gray depth image |
| CN206400641U (en) * | 2017-01-13 | 2017-08-11 | 中国人民公安大学 | Portable scene of a crime rebuilds scanner and scene of a crime three-dimensional reconstruction system |
| CN107103626A (en) * | 2017-02-17 | 2017-08-29 | 杭州电子科技大学 | A kind of scene reconstruction method based on smart mobile phone |
| CN107256540A (en) * | 2017-06-12 | 2017-10-17 | 四川大学 | Image repair method, device and three-dimensional reconstruction system |
Non-Patent Citations (1)
| Title |
|---|
| CARCERONI，R.L.，KUTULAKOS: "Multi-view scene capture by surfel sampling： From video streams to non-rigid 3D motion，shape and reflectance", 《EIGTH INTERNATIONAL CONFERENCE ON COMPUTER VISION(ICCV 2001)》 * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110634177A (en) * | 2018-06-21 | 2019-12-31 | 华为技术有限公司 | Object modeling movement method, device and equipment |
| US11436802B2 (en) | 2018-06-21 | 2022-09-06 | Huawei Technologies Co., Ltd. | Object modeling and movement method and apparatus, and device |
| CN111640175A (en) * | 2018-06-21 | 2020-09-08 | 华为技术有限公司 | Object modeling movement method, device and equipment |
| CN111640176A (en) * | 2018-06-21 | 2020-09-08 | 华为技术有限公司 | Object modeling movement method, device and equipment |
| CN108960225A (en) * | 2018-07-06 | 2018-12-07 | 四川奇迹云科技有限公司 | It is a kind of that auxiliary system is installed based on object identification and the dismantling of three-dimensional reconstruction |
| WO2020038277A1 (en) * | 2018-08-21 | 2020-02-27 | 先临三维科技股份有限公司 | Image acquisition and processing methods and apparatuses for three-dimensional scanning, and three-dimensional scanning device |
| US11887321B2 (en) | 2018-08-21 | 2024-01-30 | Shining3D Tech Co., Ltd. | Three-dimensional scanning image acquisition and processing methods and apparatuses, and three-dimensional scanning device |
| WO2020062919A1 (en) * | 2018-09-30 | 2020-04-02 | Oppo广东移动通信有限公司 | Data processing method, mec server and terminal device |
| CN109951695A (en) * | 2018-11-12 | 2019-06-28 | 北京航空航天大学 | Light field modulating three-dimensional imaging method and imaging system are moved freely based on mobile phone |
| CN109872716A (en) * | 2018-12-25 | 2019-06-11 | 南京硅基智能科技有限公司 | A kind of plant cognition voice interactive system based on two dimensional code |
| WO2020192706A1 (en) * | 2019-03-25 | 2020-10-01 | 华为技术有限公司 | Object three-dimensional model reconstruction method and device |
| CN109978931B (en) * | 2019-04-04 | 2021-12-31 | 中科海微(北京)科技有限公司 | Three-dimensional scene reconstruction method and device and storage medium |
| CN109978931A (en) * | 2019-04-04 | 2019-07-05 | 北京悉见科技有限公司 | Method for reconstructing three-dimensional scene and equipment, storage medium |
| CN110349251B (en) * | 2019-06-28 | 2020-06-16 | 深圳数位传媒科技有限公司 | Three-dimensional reconstruction method and device based on binocular camera |
| CN110349251A (en) * | 2019-06-28 | 2019-10-18 | 深圳数位传媒科技有限公司 | A kind of three-dimensional rebuilding method and device based on binocular camera |
| WO2021087800A1 (en) * | 2019-11-06 | 2021-05-14 | Oppo广东移动通信有限公司 | Information processing method, encoding apparatus, decoding apparatus, system, and storage medium |
| CN114402590A (en) * | 2019-11-06 | 2022-04-26 | Oppo广东移动通信有限公司 | Information processing method and system, encoding device, decoding device, and storage medium |
| WO2021087810A1 (en) * | 2019-11-06 | 2021-05-14 | Oppo广东移动通信有限公司 | Information processing methods and systems, and encoding apparatus, decoding apparatus and storage medium |
| CN113870338A (en) * | 2020-06-30 | 2021-12-31 | 北京瓦特曼科技有限公司 | Three-dimensional reconstruction-based zinc bath slagging-off method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108053435A (en) | Dynamic realtime three-dimensional rebuilding method and system based on handheld mobile device | |
| US10733804B2 (en) | Method and system for representing a virtual object in a view of a real environment | |
| CN109285215B (en) | Human body three-dimensional model reconstruction method and device and storage medium | |
| WO2021077720A1 (en) | Method, apparatus, and system for acquiring three-dimensional model of object, and electronic device | |
| CN108876926B (en) | Navigation method and system in panoramic scene and AR/VR client equipment | |
| CN110335343B (en) | Human body three-dimensional reconstruction method and device based on RGBD single-view-angle image | |
| US9235928B2 (en) | 3D body modeling, from a single or multiple 3D cameras, in the presence of motion | |
| CN109461180A (en) | A kind of method for reconstructing three-dimensional scene based on deep learning | |
| CN108475327A (en) | three-dimensional acquisition and rendering | |
| US20130258045A1 (en) | Method and system of spacial visualisation of objects and a platform control system included in the system, in particular for a virtual fitting room | |
| CN112037320A (en) | Image processing method, device, equipment and computer readable storage medium | |
| CN104599317A (en) | Mobile terminal and method for achieving 3D (three-dimensional) scanning modeling function | |
| Mousavi et al. | The performance evaluation of multi-image 3D reconstruction software with different sensors | |
| WO2024007478A1 (en) | Three-dimensional human body modeling data collection and reconstruction method and system based on single mobile phone | |
| CN106683163B (en) | Imaging method and system for video monitoring | |
| CN113313818A (en) | Three-dimensional reconstruction method, device and system | |
| CN110827392A (en) | Monocular image three-dimensional reconstruction method, system and device with good scene usability | |
| CN108205822B (en) | Picture pasting method and device | |
| WO2014177604A1 (en) | Method and system for generating a 3d model | |
| Sevastopolsky et al. | Relightable 3d head portraits from a smartphone video | |
| JP5059503B2 (en) | Image composition apparatus, image composition method, and image composition program | |
| Kurillo et al. | A framework for collaborative real-time 3D teleimmersion in a geographically distributed environment | |
| CN110177216A (en) | Image processing method, device, mobile terminal and storage medium | |
| CN113989434A (en) | Human body three-dimensional reconstruction method and device | |
| Fechteler et al. | Articulated 3D model tracking with on-the-fly texturing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 11-13 / F, joint headquarters building, high tech Zone, 63 Xuefu Road, Yuehai street, Nanshan District, Shenzhen, Guangdong 518000 Applicant after: Obi Zhongguang Technology Group Co., Ltd Address before: A808, Zhongdi building, industry university research base, China University of Geosciences, No.8, Yuexing Third Road, Nanshan District, Shenzhen, Guangdong 518000 Applicant before: SHENZHEN ORBBEC Co.,Ltd. |
|
| CB02 | Change of applicant information |