CN108182702B - Real-time three-dimensional modeling method and system based on depth image acquisition equipment - Google Patents
Real-time three-dimensional modeling method and system based on depth image acquisition equipment Download PDFInfo
- Publication number
- CN108182702B CN108182702B CN201611121814.6A CN201611121814A CN108182702B CN 108182702 B CN108182702 B CN 108182702B CN 201611121814 A CN201611121814 A CN 201611121814A CN 108182702 B CN108182702 B CN 108182702B
- Authority
- CN
- China
- Prior art keywords
- data
- depth
- points
- image acquisition
- dot matrix
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/10—Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Landscapes
- Physics & Mathematics (AREA)
- Geometry (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Image Processing (AREA)
Abstract
The invention belongs to the technical field of three-dimensional modeling, and particularly relates to a real-time three-dimensional modeling method and a real-time three-dimensional modeling system based on depth image acquisition equipment. The real-time three-dimensional modeling method and the system based on the depth image acquisition equipment have the advantages that the modeling process is efficient and simple, and the three-dimensional modeling can be completed in real time; high-precision modeling can be realized through the depth image acquisition equipment, and meanwhile, the surface color and texture of the object can be really restored; the realized equipment has simple structure and greatly reduces the cost of three-dimensional modeling.
Description
Technical Field
The invention relates to the technical field of three-dimensional modeling, in particular to a real-time three-dimensional modeling method and system based on depth image acquisition equipment.
Background
The application field of the object three-dimensional model is very wide, such as the fields of design simulation, virtual reality, 3D movies and the like. The existing three-dimensional modeling method comprises a binocular stereo vision technology, three-dimensional laser scanner modeling and the like. The three-dimensional modeling method based on the binocular stereo vision technology needs to perform complex operation processing such as distortion correction and stereo matching on two images with parallax, and is low in modeling speed, poor in real-time performance and low in modeling precision. The modeling method based on the three-dimensional laser scanner has the advantages of complex and expensive adopted equipment, higher cost and incapability of being widely applied.
Disclosure of Invention
Aiming at the defects in the prior art, the real-time three-dimensional modeling method and the system based on the depth image acquisition equipment have the advantages of high efficiency, simplicity, high precision and low cost, and can truly reduce the surface color and texture of an object.
In a first aspect, the invention provides a real-time three-dimensional modeling method based on a depth image acquisition device, which includes: acquiring image data and depth data of an object to be reconstructed from a plurality of angles through a plurality of depth image acquisition devices respectively; integrating all depth data according to the calibration sequence to obtain a first ordered dot matrix; extracting points belonging to the foreground image in the first ordered dot matrix;
calculating the coordinates of the points belonging to the foreground image in the first ordered dot matrix in a three-dimensional space coordinate system according to the depth data of the points, the position information of the points in the first ordered dot matrix and the space geometric relationship obtained through calibration; combining the points with the same three-dimensional space coordinates, and rearranging the first ordered dot matrix according to the point combination result to obtain a second ordered dot matrix; gridding the second ordered dot matrix to obtain unit surface and grid link data; mapping the unit surfaces according to the image data to obtain mapping data and a UV layout of each unit surface; and finally determining three-dimensional model data, wherein the three-dimensional model data comprises vertex data, the grid link data, the UV layout and the map data.
The invention provides a real-time three-dimensional modeling method based on depth image acquisition equipment, which is characterized in that image data and depth data of an object are acquired from multiple angles through multiple depth image acquisition equipment, the space coordinates of a three-dimensional model are determined through a space geometric relationship, the surface of the model is rapidly reconstructed through lattice data gridding, and the surface of the model is mapped through the image data, so that the real color and texture of the surface of the object are reduced. The real-time three-dimensional modeling method based on the depth image acquisition equipment has no complex calculation, is efficient and simple, can complete modeling in real time, and has the modeling speed of 100 times per second; high-precision modeling can be realized through the depth image acquisition equipment, and meanwhile, the surface color and texture of the object can be really restored; the realized equipment has simple structure, and the modeling cost is greatly reduced; the method of the embodiment is suitable for depth image acquisition equipment in various forms, and compared with the application range of a general modeling method, the accuracy of the depth image acquisition equipment is higher, and the accuracy of the model established by the method of the embodiment is higher.
Preferably, the extracting the points belonging to the foreground image in the first ordered lattice comprises: comparing the depth data of the points in the first ordered lattice with a first depth threshold and a second depth threshold, and if the depth data of the points is between the first depth threshold and the second depth threshold, determining that the points belong to a foreground image.
Preferably, the gridding the second ordered lattice to obtain the cell plane and the grid link data includes: numbering the points in the second ordered dot matrix, forming a unit surface by three points in a first row and a first column, a first row and a second column and a second row and a first column in the second ordered dot matrix, and generating grid link data corresponding to the unit surface, wherein the grid link data are arrays formed by numbering the three points forming the unit surface in sequence, and repeating the above steps to complete the gridding of all the points in the second ordered dot matrix and obtain the grid link data corresponding to all the unit surfaces.
Preferably, the mapping the unit surfaces according to the image data to obtain mapping data and a UV layout of each unit surface includes: and according to the three points forming the unit surface, extracting an image of a corresponding area from the image data to serve as mapping data of the unit surface, establishing an index relation between the mapping data and the unit surface, and storing the mapping data and the unit surface into a UV layout.
In a second aspect, the invention provides a real-time three-dimensional modeling system based on a depth image acquisition device, comprising: a plurality of depth image acquisition devices and processing units which are calibrated in advance; the depth image acquisition devices are all connected with the processing unit; the depth image acquisition equipment is used for acquiring image data and depth data of an object to be reconstructed from different angles and outputting the image data and the depth data to the processing unit; the processing unit is used for integrating all the depth data to obtain a first ordered dot matrix according to the calibration sequence; extracting points belonging to the foreground image in the first ordered dot matrix; calculating the coordinates of the points belonging to the foreground image in the first ordered dot matrix in a three-dimensional space coordinate system according to the depth data of the points, the position information of the points in the first ordered dot matrix and the space geometric relationship obtained through calibration; combining the points with the same three-dimensional space coordinates, and rearranging the first ordered dot matrix according to the point combination result to obtain a second ordered dot matrix; gridding the second ordered dot matrix to obtain unit surface and grid link data; mapping the unit surfaces according to the image data to obtain mapping data and a UV layout of each unit surface; and finally determining three-dimensional model data, wherein the three-dimensional model data comprises vertex data, the grid link data, the UV layout and the map data.
According to the real-time three-dimensional modeling system based on the depth image acquisition equipment, the depth image acquisition equipment does not need to be moved in the modeling process, the complexity of the system is reduced, the equipment structure is simple, and the modeling cost is greatly reduced; complex calculation does not exist, the method is efficient and simple, three-dimensional modeling can be completed in real time, and three-dimensional reconstruction can be realized for any object in the space within 100 fps; high-precision modeling can be realized through the depth image acquisition equipment, and meanwhile, the surface color and texture of the object can be really restored.
Preferably, the number of the depth image acquisition devices is 3N, N is a natural number greater than 0, every 3 depth image acquisition devices form a regular triangle at the position, N groups of the depth image acquisition devices shoot the object to be reconstructed from different angles, and the shooting directions of the 3N depth image acquisition devices all point to the object to be reconstructed.
Preferably, the number of the depth image acquisition devices is 6, wherein 3 depth image acquisition devices are located to form a regular triangle, the other 3 depth image acquisition devices are respectively located at the middle point of the side of the regular triangle, and the shooting direction of the 6 depth image acquisition devices points to the geometric center of the regular triangle.
Preferably, the two sets of depth image acquisition devices are at different heights.
Drawings
Fig. 1 is a flowchart of a real-time three-dimensional modeling method based on a depth image acquisition device according to a first embodiment of the present invention;
fig. 2 is a distribution diagram of a depth image acquisition device in a real-time three-dimensional modeling system based on the depth image acquisition device according to a third embodiment of the invention;
fig. 3 is a distribution diagram of a depth image acquisition device in a real-time three-dimensional modeling system based on the depth image acquisition device according to a fourth embodiment of the invention;
fig. 4 is a schematic diagram illustrating a method for merging multiple sets of depth data according to a first embodiment of the present invention;
FIG. 5 is a schematic diagram of a second ordered lattice meshing provided by the first embodiment of the present invention;
fig. 6 is a schematic diagram of a correspondence relationship between image data and depth data.
Detailed Description
Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and therefore are only examples, and the protection scope of the present invention is not limited thereby.
It is to be noted that, unless otherwise specified, technical or scientific terms used herein shall have the ordinary meaning as understood by those skilled in the art to which the invention pertains.
Example one
As shown in fig. 1, the real-time three-dimensional modeling method based on a depth image acquisition device provided in this embodiment includes:
and step S1, acquiring image data and depth data of the object to be reconstructed from a plurality of angles by a plurality of depth image acquisition devices respectively.
The depth image acquisition equipment is equipment capable of acquiring images and depths corresponding to the images, and an rgb-d camera (such as Kinect) can be selected and used, and can directly output image data and depth data. The depth image acquisition equipment can also adopt a mode of combining the camera and the distance measuring device, the relation between the image data output by the camera and the depth data output by the distance measuring device can be obtained through calibration, but the image data acquired by the camera and the depth data output by the distance measuring device are in the same proportion. As shown in fig. 4, the depth data acquired by each depth image acquisition device is stored in the form of a rectangular dot matrix, and each dot stores depth data. The image data and the depth data are acquired from different types of data within the same spatial range.
And step S2, integrating all the depth data according to the calibration sequence to obtain a first ordered lattice.
The calibration sequence is determined in the process of calibrating the depth image acquisition devices, so that the depth data acquired by all the depth image acquisition devices can be merged according to a real spatial position in the merging process. The calibration sequence is described by the following example, as shown in fig. 2, taking a first depth image acquisition device, a second depth image acquisition device, and a third depth image acquisition device as an example, after calibration of each device is completed, it is determined that the devices are combined in a clockwise calibration sequence, the first depth data acquired by the first depth image acquisition device is recorded, the second depth data acquired by the second depth image acquisition device is recorded, the third depth data acquired by the third depth image acquisition device is recorded, as shown in fig. 4, the first depth data is sequentially stored in the first ordered dot matrix in a sequence from right to left, the second depth data is sequentially stored in the first ordered dot matrix in a sequence from right to left, and finally the third depth data is sequentially stored in the first ordered dot matrix in a sequence from right to left.
Step S3, extracting the points belonging to the foreground image in the first ordered lattice.
Wherein, the foreground image corresponds to the object to be modeled.
And step S4, calculating the coordinates of the points belonging to the foreground image in the first ordered lattice in the three-dimensional space coordinate system according to the depth data of the points, the position information of the points in the first ordered lattice and the space geometric relationship obtained through calibration.
The three-dimensional space coordinate system is a coordinate system corresponding to the modeling model, the space geometric relationship is a mapping relationship for mapping points in the first ordered dot matrix to the three-dimensional space coordinate system, and once the space position of the depth image acquisition device is determined, the space geometric relationship can be determined through the solid geometric knowledge, and is not repeated here. The position information is represented by the horizontal and vertical coordinates of the dots in the first ordered lattice.
And step S5, merging the points with the same three-dimensional space coordinates, and rearranging the first ordered lattice to obtain a second ordered lattice according to the point merging result.
And step S6, gridding the second ordered lattice to obtain cell surface and grid link data.
And step S7, mapping the cell surface according to the image data to obtain mapping data and a UV layout of each cell surface.
And step S8, finally determining three-dimensional model data, wherein the three-dimensional model data comprises vertex data, grid link data, a UV layout and map data.
The vertex data are three-dimensional space coordinates of all vertexes of the three-dimensional model, the vertexes of the three-dimensional model correspond to the points in the second ordered dot matrix, and the three-dimensional space coordinates of all the vertexes are stored into the vertex data according to a certain sequence to form an array. The mesh link data stores three-dimensional spatial information of all the unit planes. The UV layout map is an index relationship between each cell face and the map data. The map data stores image data required for mapping.
Most three-dimensional model software can obtain a reconstructed three-dimensional model according to three-dimensional model data. Firstly, determining the positions of all vertexes of an object to be reconstructed in a three-dimensional space according to vertex data; then, connecting all vertexes into a face according to the mesh link data to obtain a white mode; finally, the chartlet data is assigned to the white mode according to the UV layout, so that the whole three-dimensional model can be displayed.
In the real-time three-dimensional modeling method based on the depth image acquisition device provided by this embodiment, the image data and the depth data of the object are acquired from multiple angles by multiple depth image acquisition devices, the spatial coordinates of the three-dimensional model are determined by the spatial geometric relationship, the model surface is rapidly reconstructed by the latticed data gridding, and the image data is used to map the model surface, so as to reduce the real color and texture of the object surface. The real-time three-dimensional modeling method based on the depth image acquisition equipment has no complex calculation, is efficient and simple, can complete modeling in real time, and has the modeling speed of 100 times per second; high-precision modeling can be realized through the depth image acquisition equipment, and meanwhile, the surface color and texture of the object can be really restored; the realized equipment has simple structure, and the modeling cost is greatly reduced; the method of the embodiment is suitable for depth image acquisition equipment in various forms, and compared with the application range of a general modeling method, the accuracy of the depth image acquisition equipment is higher, and the accuracy of the model established by the method of the embodiment is higher.
The specific implementation method that can be implemented by step S3 includes: and comparing the depth data of the points in the first ordered lattice with a first depth threshold and a second depth threshold, and if the depth data of the points is between the first depth threshold and the second depth threshold, determining that the points belong to the foreground image. In the modeling process, a shooting range is defined, an object must be placed in the shooting range, a first depth threshold value can be the distance from the depth image acquisition device to the nearest shooting range, and a second depth threshold value can be the distance from the depth image acquisition device to the farthest shooting range.
The specific implementation method of step S6 includes: and numbering the points in the second ordered dot matrix, forming a unit surface by three points in a first row and a first column, a first row and a second column and a second row and a first column in the second ordered dot matrix, and generating grid link data corresponding to the unit surface, wherein the grid link data are arrays formed by numbering the three points forming the unit surface in sequence, and so on, completing the gridding of all the points in the second ordered dot matrix, and obtaining the grid link data corresponding to all the unit surfaces. As shown in fig. 5, the dots in the second ordered dot matrix are numbered in the order from left to right and from top to bottom, and the dot 0, the dot 1, and the dot 4 form a unit plane a, then the mesh link data corresponding to the unit plane a is (0,1,4), and the mesh link data stores the numbers of the three dots in an ordered manner. Since the vertex data is also stored in the three-dimensional space coordinates of the vertices by the numbers of the points in the second ordered lattice, the three-dimensional space coordinates of the 0 th, 1 th, and 4 th positions are extracted from the vertex data based on the mesh link data (0,1,4), and the element plane a can be restored.
The specific implementation method of step S7 includes: and according to the three points forming the unit surface, extracting the image of the corresponding area from the image data as mapping data of the unit surface, establishing an index relation between the mapping data and the unit surface and storing the mapping data and the unit surface into the UV layout. As shown in fig. 6, each point constituting the unit surface has data corresponding to the point in the image data, so that once the three points of the unit surface are determined, the image corresponding to the unit surface can be located from the image data, and the image is extracted as the map data in a manner of scaling in the same scale according to the resolution required by modeling, so as to meet the requirements of different resolutions. The image data can be acquired through a single high-definition camera, but the image data acquired through the high-definition camera and the depth data output by the distance measuring device are in the same proportion, so that the accuracy of the mapping data is further improved, and the established three-dimensional model is more vivid.
Example two
Based on the same inventive concept as the real-time three-dimensional modeling method, the embodiment provides a real-time three-dimensional modeling system based on a depth image acquisition device, which includes: a plurality of depth image acquisition devices and processing units which are calibrated in advance; the depth image acquisition devices are all connected with the processing unit;
the depth image acquisition equipment is used for acquiring image data and depth data of an object to be reconstructed from different angles and outputting the image data and the depth data to the processing unit;
the processing unit is used for integrating all the depth data to obtain a first ordered dot matrix according to the calibration sequence; extracting points belonging to the foreground image in the first ordered dot matrix; calculating the coordinates of the points belonging to the foreground image in the first ordered dot matrix in a three-dimensional space coordinate system according to the depth data of the points, the position information of the points in the first ordered dot matrix and the space geometric relationship obtained through calibration; combining the points with the same three-dimensional space coordinates, and rearranging the first ordered dot matrix according to the point combination result to obtain a second ordered dot matrix; gridding the second ordered dot matrix to obtain unit surface and grid link data; mapping the unit surfaces according to the image data to obtain mapping data and a UV layout of each unit surface; and finally determining three-dimensional model data, wherein the three-dimensional model data comprises vertex data, grid link data, a UV layout and chartlet data.
Generally, the method for reconstructing the model by using the mobile device requires continuous partial recalculation of the formed model, and the mobile device has a complex structure and complicated calculation process. The real-time three-dimensional modeling system based on the depth image acquisition equipment provided by the embodiment does not need to move the depth image acquisition equipment in the modeling process, so that the complexity of the system is reduced, the structure of the equipment is simple, and the modeling cost is greatly reduced; complex calculation does not exist, the method is efficient and simple, three-dimensional modeling can be completed in real time, and three-dimensional reconstruction can be realized for any object in the space within 100 fps; high-precision modeling can be realized through the depth image acquisition equipment, and meanwhile, the surface color and texture of the object can be really restored.
EXAMPLE III
When shooting an object, a minimum of 3 cameras are needed to obtain a complete image of the object, and the 3 cameras need to be arranged in a regular triangle manner. Therefore, on the basis of the second embodiment, in order to simplify the system, reduce the cost, and ensure the modeling accuracy, the present embodiment provides another real-time three-dimensional modeling system based on depth image acquisition devices, which includes 3N depth image acquisition devices that have been calibrated in advance, and a processing unit. And the 3N depth image acquisition devices are all connected with the processing unit. N is a natural number larger than 0, a regular triangle is formed by the positions of every 3 sets of depth image acquisition equipment, the N sets of depth image acquisition equipment shoot the object to be reconstructed from different angles, and the shooting directions of the 3N sets of depth image acquisition equipment all point to the object to be reconstructed. The angle of data acquisition of each depth image acquisition device is 60 degrees. An object to be modeled is placed at the center, and an image of the object can be acquired without dead angles through the 3N depth image acquisition devices.
As shown in fig. 2, a preferred arrangement of 6 depth image acquisition devices is given, and the six-mango star arrangement reduces the number of depth image acquisition devices, reduces the data amount to be processed by the processing unit, reduces the performance requirement on the system, and further improves the modeling efficiency. A blind area may exist in one group of depth image acquisition equipment, and the blind area can be eliminated through the second group of depth image acquisition equipment, so that the finally established three-dimensional model is more complete and accurate.
In order to further improve the acquisition range and accuracy, in fig. 2, two sets of depth image acquisition devices are at different heights, one set of depth image acquisition devices is mainly used for acquiring data of the upper part of the object, and the other set of depth image acquisition devices is mainly used for acquiring data of the lower part of the object.
The processing unit is used for integrating all depth data to obtain a first ordered dot matrix according to the calibration sequence of the 3N devices; extracting points belonging to the foreground image in the first ordered dot matrix; calculating the coordinates of the points belonging to the foreground image in the first ordered dot matrix in a three-dimensional space coordinate system according to the depth data of the points, the position information of the points in the first ordered dot matrix and the space geometric relationship obtained through calibration; combining the points with the same three-dimensional space coordinates, and rearranging the first ordered dot matrix according to the point combination result to obtain a second ordered dot matrix; gridding the second ordered dot matrix to obtain unit surface and grid link data; mapping the unit surfaces according to the image data to obtain mapping data and a UV layout of each unit surface; and finally determining three-dimensional model data, wherein the three-dimensional model data comprises vertex data, grid link data, a UV layout and chartlet data.
Example four
On the basis of the second embodiment, in order to simplify the system, reduce the cost, and simultaneously ensure the modeling accuracy, as shown in fig. 3, the present embodiment provides another real-time three-dimensional modeling system based on depth image acquisition devices, which includes 6 depth image acquisition devices and a processing unit, where each depth image acquisition device is connected to the processing unit. Wherein 3 depth image acquisition equipment position constitute regular triangle, and 3 depth image acquisition equipment lie in the mid point of regular triangle's limit respectively in addition, constitute interior regular triangle. The shooting directions of the 6 depth image acquisition devices point to the geometric center of the regular triangle, and the angle of data acquired by each depth image acquisition device is 60 degrees.
Wherein, the processing steps in the processing unit are as follows:
and step S10, completing preliminary modeling according to the image data and the depth data acquired by the 3 depth image acquisition devices positioned at the vertexes of the regular triangle. The method comprises the following specific steps: integrating 3 sets of depth data according to the calibration sequence of 3 depth image acquisition devices positioned on the vertex of the regular triangle to obtain a first ordered dot matrix; extracting points belonging to the foreground image in the first ordered dot matrix; calculating the coordinates of the points belonging to the foreground image in the first ordered dot matrix in a three-dimensional space coordinate system according to the depth data of the points, the position information of the points in the first ordered dot matrix and the space geometric relationship obtained through calibration; and combining the points with the same three-dimensional space coordinates, and rearranging the first ordered dot matrix according to the point combination result to obtain a second ordered dot matrix.
And step S20, supplementing and perfecting the cavity on the surface of the model according to the data acquired by the 3 depth image acquisition devices on the sides of the regular triangle, and improving the data precision. The method comprises the following specific steps:
step S201, acquiring image data and depth data acquired by 3 depth image acquisition devices on the side of the regular triangle.
Step S202, respectively extracting points belonging to the foreground image from the 3 sets of depth data.
Step S203, calculating the coordinates of the points belonging to the foreground image in the three-dimensional space coordinate system according to the depth data of the points, the position information of the points in the depth data and the space geometric relationship obtained through calibration.
Step S204, according to the three-dimensional space coordinates of the points obtained in step S203, whether the points with the same three-dimensional space coordinates exist in the second ordered lattice in step S10 is determined.
In step S205, if there is no point with the same coordinate, the point obtained in step S203 is supplemented to the corresponding position in the second-order lattice data.
Step S206, if the point with the same coordinate exists, recording the depth data of the point in the second ordered dot matrix as depth1, recording the depth data of the point obtained in the step S201 as depth2, comparing depth1 and depth2 with a matching degree threshold respectively, and screening through comparison results; and if the screening result is that depth2 is selected, replacing depth1 in the second-order lattice data with depth2, otherwise, not modifying the second-order lattice data. The matching degree threshold value and the screening strategy can be set according to actual requirements.
Step S30, gridding the second ordered lattice to obtain cell surface and grid link data; mapping the unit surfaces according to the image data to obtain mapping data and a UV layout of each unit surface; and finally determining three-dimensional model data, wherein the three-dimensional model data comprises vertex data, grid link data, a UV layout and chartlet data.
On the basis of the second embodiment, the real-time three-dimensional modeling system based on the depth image acquisition equipment provided by the embodiment is modeled by 3 depth image acquisition equipment on the vertex of the regular triangle, and then the cavities on the surface of the model are repaired by 3 depth image acquisition equipment on the sides of the regular triangle, so that the precision of the three-dimensional model is further improved.
To further improve the acquisition range and accuracy, the 3 devices making up the outer triangle and the 3 devices making up the inner triangle are at different heights. For example, the height of 3 devices constituting the outer triangle is higher than that of 3 devices constituting the inner triangle, so that the 3 devices constituting the outer triangle are mainly used to acquire data of the upper part of the object and the 3 devices constituting the inner triangle are mainly used to acquire data of the lower part of the object. It is preferable that the height of the 3 devices constituting the outer triangle is 2 times the height of the 3 devices constituting the inner triangle.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.
Claims (7)
1. A real-time three-dimensional modeling method based on a depth image acquisition device is characterized by comprising the following steps:
acquiring image data and depth data of an object to be reconstructed from a plurality of angles through a plurality of depth image acquisition devices respectively;
integrating all depth data according to the calibration sequence to obtain a first ordered dot matrix;
extracting points belonging to the foreground image in the first ordered dot matrix;
the extracting of the points belonging to the foreground image in the first ordered lattice comprises:
comparing the depth data of the points in the first ordered lattice with a first depth threshold and a second depth threshold, and if the depth data of the points is between the first depth threshold and the second depth threshold, determining that the points belong to a foreground image;
calculating the coordinates of the points belonging to the foreground image in the first ordered dot matrix in a three-dimensional space coordinate system according to the depth data of the points, the position information of the points in the first ordered dot matrix and the space geometric relationship obtained through calibration;
combining the points with the same three-dimensional space coordinates, and rearranging the first ordered dot matrix according to the point combination result to obtain a second ordered dot matrix;
gridding the second ordered dot matrix to obtain unit surface and grid link data;
mapping the unit surfaces according to the image data to obtain mapping data and a UV layout of each unit surface;
and finally determining three-dimensional model data, wherein the three-dimensional model data comprises vertex data, the grid link data, the UV layout and the map data.
2. The method of claim 1, wherein the gridding the second ordered lattice to obtain cell plane and lattice link data comprises:
numbering the points in the second ordered dot matrix, forming a unit surface by three points in a first row and a first column, a first row and a second column and a second row and a first column in the second ordered dot matrix, and generating grid link data corresponding to the unit surface, wherein the grid link data are arrays formed by numbering the three points forming the unit surface in sequence, and repeating the above steps to complete the gridding of all the points in the second ordered dot matrix and obtain the grid link data corresponding to all the unit surfaces.
3. The method of claim 2, wherein said mapping said cell faces according to said image data to obtain mapping data and a UV layout map for each cell face comprises: and according to the three points forming the unit surface, extracting an image of a corresponding area from the image data to serve as mapping data of the unit surface, establishing an index relation between the mapping data and the unit surface, and storing the mapping data and the unit surface into a UV layout.
4. A real-time three-dimensional modeling system based on a depth image acquisition device, comprising: a plurality of depth image acquisition devices and processing units which are calibrated in advance;
the depth image acquisition devices are all connected with the processing unit;
the depth image acquisition equipment is used for acquiring image data and depth data of an object to be reconstructed from different angles and outputting the image data and the depth data to the processing unit;
the processing unit is used for integrating all the depth data to obtain a first ordered dot matrix according to the calibration sequence; extracting points belonging to the foreground image in the first ordered dot matrix; calculating the coordinates of the points belonging to the foreground image in the first ordered dot matrix in a three-dimensional space coordinate system according to the depth data of the points, the position information of the points in the first ordered dot matrix and the space geometric relationship obtained through calibration; combining the points with the same three-dimensional space coordinates, and rearranging the first ordered dot matrix according to the point combination result to obtain a second ordered dot matrix; gridding the second ordered dot matrix to obtain unit surface and grid link data; mapping the unit surfaces according to the image data to obtain mapping data and a UV layout of each unit surface; and finally determining three-dimensional model data, wherein the three-dimensional model data comprises vertex data, the grid link data, the UV layout and the map data.
5. The system according to claim 4, wherein the number of the depth image capturing devices is 3N, N is a natural number greater than 0, every 3 depth image capturing devices are located to form a regular triangle, N groups of the depth image capturing devices capture the object to be reconstructed from different angles, and the capturing directions of the 3N depth image capturing devices all point to the object to be reconstructed.
6. The system according to claim 4, wherein the number of the depth image acquisition devices is 6, wherein 3 depth image acquisition devices are located to form a regular triangle, the other 3 depth image acquisition devices are respectively located at the middle points of the sides of the regular triangle, and the shooting directions of the 6 depth image acquisition devices point to the geometric center of the regular triangle.
7. The system of claim 6, wherein the two sets of depth image acquisition devices are at different heights.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611121814.6A CN108182702B (en) | 2016-12-08 | 2016-12-08 | Real-time three-dimensional modeling method and system based on depth image acquisition equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611121814.6A CN108182702B (en) | 2016-12-08 | 2016-12-08 | Real-time three-dimensional modeling method and system based on depth image acquisition equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108182702A CN108182702A (en) | 2018-06-19 |
| CN108182702B true CN108182702B (en) | 2020-06-30 |
Family
ID=62544814
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611121814.6A Active CN108182702B (en) | 2016-12-08 | 2016-12-08 | Real-time three-dimensional modeling method and system based on depth image acquisition equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108182702B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109389665B (en) * | 2018-08-24 | 2021-10-22 | 先临三维科技股份有限公司 | Texture obtaining method, device and equipment of three-dimensional model and storage medium |
| CN111179332B (en) * | 2018-11-09 | 2023-12-19 | 北京市商汤科技开发有限公司 | Image processing method and device, electronic equipment and storage medium |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104599314A (en) * | 2014-06-12 | 2015-05-06 | 深圳奥比中光科技有限公司 | Three-dimensional model reconstruction method and system |
| CN104126989B (en) * | 2014-07-30 | 2016-06-01 | 福州大学 | A kind of based on the foot surfaces 3 D information obtaining method under multiple stage RGB-D pick up camera |
| US9928645B2 (en) * | 2015-04-17 | 2018-03-27 | Microsoft Technology Licensing, Llc | Raster-based mesh decimation |
| CN105045389B (en) * | 2015-07-07 | 2018-09-04 | 深圳水晶石数字科技有限公司 | A kind of demenstration method of interactive sand table system |
| CN106204727A (en) * | 2016-07-11 | 2016-12-07 | 北京大学深圳研究生院 | The method and device that a kind of foot 3-D scanning is rebuild |
-
2016
- 2016-12-08 CN CN201611121814.6A patent/CN108182702B/en active Active
Non-Patent Citations (1)
| Title |
|---|
| 基于Kinect的三维重建方法的研究与实现;李国镇;《中国优秀硕士学位论文全文数据库 信息科技辑》;20121031;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108182702A (en) | 2018-06-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104537707B (en) | Image space type stereoscopic vision moves real-time measurement system online | |
| CN109712232B (en) | Object surface contour three-dimensional imaging method based on light field | |
| CN112132972A (en) | Three-dimensional reconstruction method and system for fusing laser and image data | |
| CN111028350B (en) | Method for constructing grid map by using binocular stereo camera | |
| EP3586269A1 (en) | Image reconstruction for virtual 3d | |
| CN113192179B (en) | Three-dimensional reconstruction method based on binocular stereo vision | |
| CN109559349A (en) | A kind of method and apparatus for calibration | |
| CN109147025B (en) | RGBD three-dimensional reconstruction-oriented texture generation method | |
| CN106023230B (en) | A kind of dense matching method of suitable deformation pattern | |
| CN103606151A (en) | A wide-range virtual geographical scene automatic construction method based on image point clouds | |
| CN108369639A (en) | Use the image rendering method and system based on image of polyphaser and depth camera array | |
| CN110337674A (en) | Three-dimensional rebuilding method, device, equipment and storage medium | |
| CN108182702B (en) | Real-time three-dimensional modeling method and system based on depth image acquisition equipment | |
| CN107958489B (en) | Curved surface reconstruction method and device | |
| CN114549772A (en) | Multi-source three-dimensional model fusion processing method and system based on engineering independent coordinate system | |
| WO2018052100A1 (en) | Image processing device, image processing method, and image processing program | |
| CN104634323B (en) | A kind of multistage shooting tri-dimensional photographing system and method | |
| CN108510558B (en) | Compression method, device and the terminal of point cloud data | |
| CN115830135A (en) | Image processing method and device and electronic equipment | |
| JP7432793B1 (en) | Mapping methods, devices, chips and module devices based on three-dimensional point clouds | |
| CN104796624A (en) | Method for editing and propagating light fields | |
| GB2569609A (en) | Method and device for digital 3D reconstruction | |
| CN112116646A (en) | Light field image depth estimation method based on depth convolution neural network | |
| CN113724365A (en) | Three-dimensional reconstruction method and device | |
| CN116758206A (en) | Vector data fusion rendering method and device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |