CN117036444A - Three-dimensional model output method, device, equipment and computer readable storage medium - Google Patents
Three-dimensional model output method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN117036444A CN117036444A CN202311290411.4A CN202311290411A CN117036444A CN 117036444 A CN117036444 A CN 117036444A CN 202311290411 A CN202311290411 A CN 202311290411A CN 117036444 A CN117036444 A CN 117036444A
- Authority
- CN
- China
- Prior art keywords
- dimensional model
- voxel grid
- initial
- target image
- grid
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000004891 communication Methods 0.000 claims description 13
- 238000005070 sampling Methods 0.000 claims description 13
- 238000009877 rendering Methods 0.000 claims description 11
- 238000006243 chemical reaction Methods 0.000 claims description 9
- 238000010276 construction Methods 0.000 claims description 9
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 description 6
- 238000013461 design Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 239000011521 glass Substances 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000013144 data compression Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
The embodiment of the application relates to the technical field of three-dimensional reconstruction, and discloses a three-dimensional model output method, device, equipment and a computer readable storage medium.
Description
Technical Field
The embodiment of the application relates to the technical field of three-dimensional reconstruction, in particular to a three-dimensional model output method, a device, equipment and a computer readable storage medium.
Background
With the development of modern technology, three-dimensional models are needed in more and more aspects, such as printing parts by utilizing the three-dimensional models in industry, building design and planning by using the three-dimensional models in construction, and the like, the three-dimensional models are often manually constructed and generated by technicians according to pre-designed drawings through modeling software, and when the three-dimensional models are needed to be used for checking the built three-dimensional models on other devices, complete three-dimensional model files are needed to be sent to the other devices in a hardware copy or wireless transmission mode, and the three-dimensional models can be checked on the other devices.
However, in some application fields, the existing manner of manually constructing the three-dimensional model and directly outputting the three-dimensional model data when needed may cause inconvenience, for example, when a person in walking wants to check the building outline and road outline of an area where the person is about to arrive ahead through AR glasses or a mobile phone to distinguish whether the person is a destination, if all the buildings and roads are manually modeled by a technician in advance, a great deal of time and effort are required, and meanwhile, the data amount of the three-dimensional model is also caused to be large, and the three-dimensional model is limited by the data transmission speed when being transmitted to a terminal carried by a user for the user to read and check, so that the user may need to spend a great deal of time waiting for the three-dimensional model data to be transmitted to the user terminal. If the user walks in an area where three-dimensional modeling is not completed in advance, or if the area construction and the road change, the user cannot view or receive wrong three-dimensional model information.
Disclosure of Invention
In view of the above, embodiments of the present application provide a three-dimensional model output method, apparatus, device, and computer readable storage medium, for solving the problem of inconvenience in three-dimensional model image generation and output.
According to an aspect of an embodiment of the present application, there is provided a three-dimensional model output method including: acquiring a plurality of first initial images shot by cameras with different camera poses;
performing three-dimensional reconstruction according to the plurality of first initial images to obtain a three-dimensional model;
projecting the three-dimensional model to obtain a target image on a virtual screen;
and outputting the target image.
In an alternative manner, three-dimensional reconstruction is performed according to a plurality of first initial images, so as to obtain a three-dimensional model, which includes: generating a point cloud set according to the plurality of first initial images; constructing a three-dimensional model according to the point cloud set;
projecting the three-dimensional model to obtain a target image on a virtual screen, comprising: calculating the surface normal vector of the point cloud set; a random sampling point cloud is obtained by adopting a random algorithm aiming at the point cloud set; generating a target space voxel grid according to the surface normal vector and the random sampling point cloud; performing differential rendering on the target space voxel grid to obtain a color value of each pixel in the target space voxel grid; and projecting the color value of each pixel in the voxel grid of the target space onto the virtual screen through the projection matrix to obtain a target image.
In an alternative manner, generating a target spatial voxel grid from a surface normal vector and a random sampling point cloud, includes: combining the point cloud set with a surface normal vector to generate a surface grid; constructing an empty space voxel grid; projecting the surface grid into an empty space voxel grid to obtain an initial space voxel grid; calculating a depth value of each voxel grid in the initial spatial voxel grid; and applying the depth value to the initial space voxel grid to obtain a target space voxel grid.
In an alternative, the formula is according to:calculating a depth value of each voxel grid in the initial spatial voxel grid, wherein +_>For the depth value of a voxel grid with coordinates (x, y, z) in the initial spatial voxel grid, n is the number of triangles projected onto the voxel grid by the surface grid, < >>For a default depth value calculated from the position of the voxel grid in space,/i>Depth values of triangles projected for a surface grid onto the voxel grid, +.>Is the set of coordinates for all voxel grids in the initial spatial voxel grid.
In an alternative, the formula is according to:projecting color values of each pixel in the target spatial voxel grid into the virtual screen, wherein u, v and w are coordinate positions of a pixel in the spatial voxel grid with coordinates (X, Y, Z) in the virtual screen, < >>And->For focal length->And->Is the center point coordinates of the image in the virtual screen.
In an alternative manner, before outputting the target image, the three-dimensional model output method further includes: the coordinates of each pixel on the target image are converted into a two-dimensional coordinate system.
In an alternative manner, after outputting the target image, the three-dimensional model output method further includes: acquiring a second initial image; superposing the second initial image and the target image to obtain a new target image; and outputting the new target image.
According to another aspect of an embodiment of the present application, there is provided a three-dimensional model output device including: a first acquisition module for acquiring a plurality of first initial images photographed by cameras of different camera positions; the construction module is used for carrying out three-dimensional reconstruction according to the plurality of first initial images to obtain a three-dimensional model; the conversion module is used for projecting the three-dimensional model to obtain a target image on the virtual screen; and the first output module is used for outputting the target image.
According to another aspect of an embodiment of the present application, there is provided a three-dimensional model output apparatus including: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface are communicated with each other through the communication bus; the memory is used for storing at least one program, and the program enables the processor to execute the operation of the three-dimensional model output method according to any one of the above.
According to still another aspect of the embodiments of the present application, there is provided a computer-readable storage medium having stored therein executable instructions for causing a three-dimensional model output device to perform the operations of the three-dimensional model output method of any one of the above.
The foregoing description is only an overview of the technical solutions of the embodiments of the present application, and may be implemented according to the content of the specification, so that the technical means of the embodiments of the present application can be more clearly understood, and the following specific embodiments of the present application are given for clarity and understanding.
Drawings
The drawings are only for purposes of illustrating embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a schematic flow chart of a three-dimensional model output method according to an embodiment of the present application;
FIG. 2 is a flow chart of the sub-steps included in the steps 120 and 130 of the present application;
FIG. 3 is a flow chart of sub-steps included in step 133 of the present application;
FIG. 4 is a flow chart illustrating steps prior to step 140 of the present application;
FIG. 5 is a flow chart illustrating steps subsequent to step 140 of the present application;
FIG. 6 is a functional block diagram of a three-dimensional model output device according to an embodiment of the present application;
fig. 7 is a schematic structural diagram of a three-dimensional model output device according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be embodied in various forms and should not be limited to the embodiments set forth herein.
The three-dimensional model is a polygonal representation of an object, typically displayed by a computer or other video device, which may be a real world entity or a fictitious object. Anything that exists in physical nature can be represented by a three-dimensional model. Because of the characteristics of the three-dimensional model, the three-dimensional model can be utilized to simulate, examine and produce more conveniently in many fields, and by generating the three-dimensional model in advance, before the product is put into production, designers and decision-making staff can know the appearance and the structure of the product accurately in advance, and correct the design, the structure and the like of the product, so that the defect of the product after the product is put into production is avoided. Meanwhile, in some building engineering fields, parameters such as volume, quality and the like can be calculated in advance through analyzing the three-dimensional model, so that engineering cost can be estimated accurately, and certain convenience can be brought to operation.
At present, along with the development of science and technology, the three-dimensional model can be applied to aspects of daily life of people, people can view the pre-constructed three-dimensional model on various mobile equipment, and people can enjoy convenience brought by the three-dimensional model containing rich information anytime and anywhere, so that convenience is brought to daily activities such as map navigation, furniture arrangement, online shopping and the like of people.
However, the inventor notices that the three-dimensional model usually needs professional technicians to design and model in advance, and can be put into use after the three-dimensional model is finally completed, the construction speed of the three-dimensional model is usually slower, if the three-dimensional model needs to cover things of a larger scene, the whole three-dimensional model data will be larger, more time is likely to be consumed in the process of data transmission and loading, the three-dimensional model of the non-constructed real scene cannot be checked by a user when being reflected in the actual application scene, and more time is required to be taken for downloading the three-dimensional model data when the three-dimensional model of the constructed real scene of the larger area is required to be checked, meanwhile, the situation that the three-dimensional model cannot be loaded or the clamping occurs in the process of checking the three-dimensional model is also likely to occur, and the popularization and the use of the three-dimensional model technology are not facilitated. Therefore, there is an urgent need to develop a three-dimensional model output method that is rapid in model generation and small in output data amount.
Based on the research, the inventor designs a three-dimensional model output method, by acquiring initial images of a plurality of target objects or target areas to perform three-dimensional reconstruction, establishing a three-dimensional model for the target objects or target areas, projecting the generated three-dimensional model on a virtual screen, and outputting the image on the virtual screen obtained by projection, the generation of the three-dimensional model can be faster, a technician is not required to manually construct the model, the image output data amount is less, and the output speed is faster.
Fig. 1 shows a flowchart of a three-dimensional model output method provided by an embodiment of the present application, where the method is performed by a three-dimensional model output device, and the three-dimensional model output device may be, for example, a mobile phone, a computer, a display, and the like. As shown in fig. 1, the method comprises the steps of:
step 110: acquiring a plurality of first initial images shot by cameras with different camera poses;
step 120: performing three-dimensional reconstruction according to the plurality of first initial images to obtain a three-dimensional model;
step 130: projecting the three-dimensional model to obtain a target image on a virtual screen;
step 140: and outputting the target image.
In steps 110 to 120, the manner of acquiring the plurality of first initial images captured by the cameras with different camera poses may be through unmanned aerial vehicle aerial photography or through a mobile phone lens, where, because the plurality of first initial images are to be used for three-dimensional reconstruction, when the plurality of target images need to be adopted to generate the three-dimensional model of the same target, the plurality of first initial images obtained by capturing should at least include a part of the same target, and in some application scenarios, a single image may be adopted to generate a single three-dimensional model, at this time, the plurality of first initial images obtained by capturing may be images with completely different targets, but with joined or approximately joined boundaries, and then the plurality of three-dimensional models generated in step 120 are spliced to obtain a complete three-dimensional model.
In order to make the accuracy of the three-dimensional model obtained by three-dimensional reconstruction higher, the plurality of first initial images should be capable of reflecting information of a plurality of angles of the target object or region, and when the camera pose of the camera for shooting the first initial images is different, meaning that the shooting angles are different, it is ensured to a certain extent that the obtained plurality of first initial images can better reflect information of a plurality of angles of the target object or region. It should be noted that, the plurality of first initial images captured by the cameras with different camera positions may refer to a plurality of images captured by the same image capturing device in sequence with different camera positions, or may be a plurality of images captured by the image capturing devices with different camera positions deployed simultaneously, and a person skilled in the art should adjust a capturing policy according to actual needs to obtain a plurality of first initial images that are convenient for three-dimensional reconstruction.
In step 120, the three-dimensional reconstruction may be performed according to the plurality of first initial images, for example, by generating a point cloud according to the plurality of initial images, and then processing the point cloud to finally generate a three-dimensional model. According to practical situations, other similar modes or algorithms can be adopted for three-dimensional reconstruction, and the purpose of the three-dimensional reconstruction is to recover depth information in a plurality of first initial images as accurately as possible, so that a three-dimensional model obtained through final reconstruction is consistent with a real scene or a required target shape as much as possible.
In step 130, the three-dimensional model is projected, which means that the three-dimensional model is mapped on a certain medium to implement data compression, and in the embodiment of the present application, the three-dimensional model is projected on a virtual screen, for example, the three-dimensional model may be used to obtain a two-dimensional plane image capable of reflecting observation of the three-dimensional model from a certain angle, and the three-dimensional model may be implemented by vertically mapping all pixels on a certain angle of the three-dimensional model on a virtual screen. It should be noted that, the virtual screen may be a plane, or may be an arc surface, or even be a hemispherical surface capable of covering a larger angle of the three-dimensional model, and according to different display devices for displaying the target image that need to be output, the virtual screen may correspondingly adopt different sizes and shapes, so that the size, the dimension, the visual angle, and the like of the target image obtained on the virtual screen after the three-dimensional model is projected can meet the requirements of the display device.
It should be noted that the position of the virtual screen relative to the three-dimensional model should be changeable according to the actual output requirement. Because a general display device can only view one angle of the three-dimensional model when viewing the three-dimensional model, when a user adjusts the display angle of the three-dimensional model through gestures, keys or a controller, the position of the virtual screen can respond to the control operation of the user to perform corresponding position change, so that a target image obtained by projecting the three-dimensional model on the virtual screen can also be correspondingly changed.
In step 140, the output object of the target image may be, for example, a display screen, a mobile phone, or AR glasses. Generally, when the output object of the target image is an apparatus such as AR glasses, which has the capability of viewing a large-sized and large-angle image, the size of the virtual screen in step 130 may be larger, for example, a larger-angle area surrounding the three-dimensional model, so that the target image projected by the three-dimensional model can reflect more information of the three-dimensional model.
The three-dimensional model is generated more rapidly by acquiring a plurality of first initial images shot by cameras with different camera poses, performing three-dimensional reconstruction according to the plurality of first initial images to obtain a three-dimensional model, projecting the three-dimensional model to obtain a target image on a virtual screen, and using the target image for output.
Referring to fig. 2, fig. 2 is a flow chart illustrating the sub-steps included in the steps 120 and 130 of the present application. As shown in fig. 2, in some embodiments of the application, step 120 includes:
step 121: generating a point cloud set according to the plurality of first initial images;
step 122: constructing a three-dimensional model according to the point cloud set;
step 130 includes:
step 131: calculating the surface normal vector of the point cloud set;
step 132: a random sampling point cloud is obtained by adopting a random algorithm aiming at the point cloud set;
step 133: generating a target space voxel grid according to the surface normal vector and the random sampling point cloud;
step 134: performing differential rendering on the target space voxel grid to obtain a color value of each pixel in the target space voxel grid;
step 135: and projecting the color value of each pixel in the voxel grid of the target space onto the virtual screen through the projection matrix to obtain a target image.
In steps 121 to 122, the point cloud set refers to a set composed of a plurality of three-dimensional point clouds. The method for generating the point cloud set according to the plurality of first initial images can be used for generating a plurality of three-dimensional point clouds by calculating the characteristic of each pixel point in the first initial images, matching and corresponding the pixel points in the plurality of first initial images based on the pose of the plurality of cameras when the plurality of cameras shoot the first initial images, and constructing the pixel points in the plurality of first initial images in the same coordinate system.
In steps 131 to 133, the surface normal vector of the point cloud set is calculated, random search is performed around the point cloud set to obtain a random sampling point cloud, and then the random sampling point cloud and the surface normal vector are combined to obtain the target space voxel grid. The purpose of calculating the surface normal vector of the point cloud is to enable the inclusion of illumination and shadow data in the target spatial voxel grid generated in step 133 to enhance the effect of the finally generated three-dimensional model. The function of calculating the random sampling point cloud is to reduce the density of the point cloud under the condition of keeping the overall geometric characteristics of the point cloud unchanged so as to reduce the data quantity to be processed and improve the calculation speed.
In step 134, differential rendering is performed on the target spatial voxel grid to obtain a color value for each pixel in the target spatial voxel grid, where differential rendering refers to a rendering process that may perform differential derivation.
Through the steps 121 to 135, the three-dimensional reconstruction effect is better, the calculation speed is faster, and the parameters can be optimized in the three-dimensional reconstruction process due to the adoption of differential rendering, so that the more accurate three-dimensional reconstruction effect can be achieved.
Referring to fig. 3, fig. 3 is a flow chart illustrating the sub-steps included in step 133 of the present application. As shown in fig. 3, in one embodiment of the present application, step 133 includes:
step 1331: combining the point cloud set with a surface normal vector to generate a surface grid;
step 1332: constructing an empty space voxel grid;
step 1333: projecting the surface grid into an empty space voxel grid to obtain an initial space voxel grid;
step 1334: calculating a depth value of each voxel grid in the initial spatial voxel grid;
step 1335: and applying the depth value to the initial space voxel grid to obtain a target space voxel grid.
Specifically, in steps 1131 to 1132, the point cloud set is combined with the surface normal vector to generate a surface grid without depth value data, and an empty space voxel grid is constructed, in steps 1333 to 1334, each triangle patch in the surface grid is projected into the empty space voxel grid, the voxel grid covered by the obtained initial space voxel grid is marked as a covered state, at this time, for the voxel grid marked as the covered state, the depth value of the center point is calculated, and the calculated depth value is taken as the depth value of all triangle patches projected onto the marked voxel grid in the surface grid, and for the voxel grid not marked as the covered state, the depth value can be calculated according to the position of the voxel grid in space.
In steps 1331 to 1335, a method of generating a target spatial voxel grid is provided, in which a surface grid generated by combining a point cloud set and a surface normal vector is projected into an empty spatial voxel grid to obtain an initial spatial voxel grid, a depth value of each voxel grid in the initial spatial voxel grid is calculated, and the depth value is applied to the initial spatial voxel grid to obtain the target spatial voxel grid for generating a three-dimensional model.
In some embodiments of the present application, step 1334 is calculated as follows: according to the formula:
calculating a depth value of each voxel grid in the initial spatial voxel grid, wherein +_>For the depth value of a voxel grid with coordinates (x, y, z) in the initial spatial voxel grid, n is the number of triangles projected onto the voxel grid by the surface grid, < >>For a default depth value calculated from the position of the voxel grid in space,/i>Depth values of triangles projected for a surface grid onto the voxel grid, +.>Is the set of coordinates for all voxel grids in the initial spatial voxel grid.
The depth value of each voxel grid in the initial space voxel grid is calculated through the formula, the depth value can be applied to the initial space voxel grid to obtain the target space voxel grid containing the depth value information of each voxel grid, the reconstruction and rendering of the three-dimensional model are convenient to follow-up, the distance between each pixel and the projection view angle can be embodied in the generated three-dimensional model, the three-dimensional model is finer, and the user can observe and view conveniently.
In some embodiments of the present application, step 135 is calculated by: according to the formula:
projecting color values of each pixel in the target spatial voxel grid into the virtual screen, wherein u, v and w are coordinate positions of a pixel in the spatial voxel grid with coordinates (X, Y, Z) in the virtual screen, < >>And->For focal length->And->Is the center point coordinates of the image in the virtual screen.
If the virtual screen is a two-dimensional plane, when the three-dimensional coordinates of the pixels in the voxel grid in the target space are projected onto the virtual screen, the projected coordinates may be converted into two-dimensional coordinates on the screen coordinate system in some cases. It can be appreciated that, a person skilled in the art can adaptively convert the projection coordinates of the voxel grid in the target space on the virtual screen according to actual needs, including, but not limited to, conversion from a three-dimensional coordinate system to a two-dimensional coordinate system and conversion from an origin of the coordinate system, so as to achieve the effects of reducing the data volume and improving the calculation speed.
The projection coordinates of the color value of each pixel in the target space voxel grid in the virtual screen are calculated through the formula, so that the projection positions of each pixel in the target space voxel grid in the virtual screen can be unified on the virtual screen, and the problem of coordinate axis disorder is not easy to occur.
Referring to fig. 4, fig. 4 is a flowchart illustrating steps before step 140 of the present application. As shown in fig. 4, in some embodiments of the present application, before step 140, the method further includes:
step 1401: the coordinates of each pixel on the target image are converted into a two-dimensional coordinate system.
In some cases, for example, coordinates of several pixels obtained after the pixels in the voxel grid in the target space are projected on the virtual screen are respectively: (3, 15, 2) (3, 16, 2.) (5,3,2) it is easy to know that the virtual screen is a plane with the Z axis being 2, and at this time, the plane with the Z axis being 2 is taken as a plane coordinate system, coordinates of a plurality of pixels obtained after projection are: (3, 15) (3, 16)... (5,3).
The coordinate data of the pixel coordinate values of the target image obtained after projection on the virtual screen are more refined by converting the coordinate of each pixel on the target image into a two-dimensional coordinate system, so that the target image can be conveniently output.
As shown in fig. 5, fig. 5 is a flowchart illustrating steps after step 140 of the present application, and in some embodiments of the present application, after step 140, further includes:
step 1410: acquiring a second initial image;
step 1420: superposing the second initial image and the target image to obtain a new target image;
step 1430: and outputting the new target image.
In step 1410, the second initial image may be an image captured by a camera or a photographing device such as a mobile phone.
In steps 1420 to 1430, overlapping the second initial image with the target image to obtain a new target image means that the pixels in the second initial image are overlapped with the pixels in the target image, and the overlapping manner may be that the pixels in the second initial image and the pixels in the target image are subjected to weighted average or transparency mixing by using an image processing technology in computer vision, so as to obtain the new target image, and the new target image is output to the display device.
The second initial image is acquired, and the second initial image and the original target image are overlapped to obtain a new target image for output, so that when a target object or region corresponding to the three-dimensional model changes, the output image can reflect the change to a certain extent, a user can distinguish the difference between the generated three-dimensional model and the actual target object or region directly according to the output new target image, and adjustment of operation or road planning is correspondingly made.
Fig. 6 shows a functional block diagram of a three-dimensional model output device 600 according to one embodiment of the application. As shown in fig. 6, the three-dimensional model output device 600 includes: a first acquisition module 601, a construction module 602, a first conversion module 603 and a first output module 604. The first acquisition module 601 is used for acquiring a plurality of first initial images shot by cameras with different camera poses; the construction module 602 is configured to perform three-dimensional reconstruction according to the plurality of first initial images to obtain a three-dimensional model; the first conversion module 603 is configured to project the three-dimensional model to obtain a target image on the virtual screen; the first output module 604 is configured to output a target image.
In some embodiments, the build module 602 further includes a first generation unit and a second generation unit. The first generating unit is used for generating point clouds according to the plurality of first initial images, and the second generating unit is used for constructing a three-dimensional model according to the point clouds.
In some embodiments, the first conversion module 603 further includes a first calculation unit, a second calculation unit, a third generation unit, a rendering unit, and a projection unit. The first computing unit is used for computing the surface normal vector of the point cloud, the second computing unit is used for obtaining random sampling point cloud by adopting a random algorithm aiming at the point cloud, the third generating unit is used for generating a target space voxel grid according to the surface normal vector and the random sampling point cloud, the rendering unit is used for conducting differential rendering on the target space voxel grid to obtain the target space voxel grid, and the projection unit is used for projecting the color value of each pixel in the target space voxel grid onto a virtual screen through a projection matrix to obtain a target image.
In some embodiments, the third generation unit further comprises a generation element, a construction element, a projection element, a calculation element, and a processing element. The generating element is used for combining the point cloud set with the surface normal vector to generate a surface grid, the constructing element is used for constructing an empty space voxel grid, the projection element is used for projecting the surface grid into the empty space voxel grid to obtain an initial space voxel grid, the calculating element is used for calculating the depth value of each voxel grid in the initial space voxel grid, and the processing element is used for applying the depth value into the initial space voxel grid to obtain a target space voxel grid.
In some embodiments, the three-dimensional model output device 600 further includes a second conversion module. The second conversion module is used for converting the coordinates of each pixel on the target image into a two-dimensional coordinate system.
In some embodiments, the three-dimensional model output device 600 further includes a second acquisition module, a superposition module, and a second output module. The second acquisition module is used for acquiring a second initial image, the superposition module is used for superposing the second initial image and the target image to obtain a new target image, and the second output module is used for outputting the new target image.
According to another aspect of an embodiment of the present application, there is provided a three-dimensional model output device. Referring specifically to fig. 7, fig. 7 is a schematic structural diagram of a three-dimensional model output device according to an embodiment of the present application, and the embodiment of the present application is not limited to the specific implementation of the three-dimensional model output device.
As shown in fig. 7, the three-dimensional model output device may include: a processor 702, a memory 706, a communication interface 704, and a communication bus 708.
Wherein the processor 702, the memory 706, and the communication interface 704 communicate with each other via a communication bus 708. The memory 706 is configured to store at least one program 710, and the program 710 causes the processor 702 to perform the relevant steps in the three-dimensional model output method embodiment described above.
In particular, program 710 may include program code including computer-executable instructions.
The processor 702 may be a Central Processing Unit (CPU), or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present application. The three-dimensional model output device includes one or more processors, which may be the same type of processor, such as one or more CPUs; but may also be different types of processors such as one or more CPUs and one or more ASICs.
Memory 706 for storing a program 710. The memory 706 may comprise high-speed RAM memory or may further comprise non-volatile memory (non-volatile memory), such as at least one disk memory.
The program 710 may be specifically invoked by the processor 702 to cause the three-dimensional model output device to: acquiring a plurality of first initial images shot by cameras with different camera poses; performing three-dimensional reconstruction according to the plurality of first initial images to obtain a three-dimensional model; projecting the three-dimensional model to obtain a target image on a virtual screen; and outputting the target image.
The embodiment of the application also provides a computer readable storage medium, wherein executable instructions are stored in the storage medium, and when the executable instructions run on the three-dimensional model output equipment, the three-dimensional model output equipment executes the three-dimensional model output method in any embodiment;
the executable instructions may be specifically operable to cause the three-dimensional model output device to: acquiring a plurality of first initial images shot by cameras with different camera poses; performing three-dimensional reconstruction according to the plurality of first initial images to obtain a three-dimensional model; projecting the three-dimensional model to obtain a target image on a virtual screen; and outputting the target image.
The algorithms or displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with the teachings herein. The required structure for a construction of such a system is apparent from the description above. In addition, embodiments of the present application are not directed to any particular programming language. It will be appreciated that the teachings of the present application described herein may be implemented in a variety of programming languages, and the above description of specific languages is provided for disclosure of enablement and best mode of the present application.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the above description of exemplary embodiments of the application, various features of the embodiments of the application are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component, and they may be divided into a plurality of sub-modules or sub-units or sub-components. Any combination of all of the features disclosed in this specification (including the accompanying abstract and drawings), and all of the processes or units of any method or apparatus so disclosed, may be used in combination, except insofar as at least some of such features and/or processes or units are mutually exclusive. Each feature disclosed in this specification (including the accompanying abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.
It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the application. The steps in the above embodiments should not be construed as limiting the order of execution unless specifically stated.
Claims (10)
1. A three-dimensional model output method, comprising:
acquiring a plurality of first initial images shot by cameras with different camera poses;
performing three-dimensional reconstruction according to the plurality of first initial images to obtain a three-dimensional model;
projecting the three-dimensional model to obtain a target image on a virtual screen;
and outputting the target image.
2. The method according to claim 1, wherein the performing three-dimensional reconstruction from the plurality of first initial images to obtain a three-dimensional model includes:
generating a point cloud set according to a plurality of first initial images;
constructing the three-dimensional model according to the point cloud set;
the projecting the three-dimensional model to obtain a target image on a virtual screen includes:
calculating the surface normal vector of the point cloud set;
a random sampling point cloud is obtained by adopting a random algorithm aiming at the point cloud set;
generating a target space voxel grid according to the surface normal vector and the random sampling point cloud;
performing differential rendering on the target space voxel grid to obtain a color value of each pixel in the target space voxel grid;
and projecting the color value of each pixel in the target space voxel grid onto the virtual screen through a projection matrix to obtain the target image.
3. The method of claim 2, wherein generating a target spatial voxel grid from the surface normal vector and the random sampling point cloud comprises:
combining the point cloud set with the surface normal vector to generate a surface grid;
constructing an empty space voxel grid;
projecting the surface grid into the empty space voxel grid to obtain an initial space voxel grid;
calculating a depth value of each voxel grid in the initial spatial voxel grid;
and applying the depth value to the initial space voxel grid to obtain a target space voxel grid.
4. A three-dimensional model output method according to claim 3, characterized by the formula:calculating a depth value of each voxel grid in the initial spatial voxel grid, wherein +.>For the depth value of a voxel grid with coordinates (x, y, z) in the initial spatial voxel grid, n is the number of triangles projected onto the voxel grid by the surface grid,/for the voxel grid with coordinates (x, y, z)>For a default depth value calculated from the position of the voxel grid in space,/i>For the depth values of triangles projected by the surface mesh onto the voxel grid, +.>Is a set of coordinates for all voxel grids in the initial spatial voxel grid.
5. The three-dimensional model output method according to claim 2, wherein according to the formula:projecting color values of each pixel in the target spatial voxel grid into the virtual screen, wherein u, v, and w are coordinate positions in the virtual screen of a pixel in the spatial voxel grid having coordinates (X, Y, Z), and wherein u, v, and w are the coordinate positions in the virtual screen of the pixel in the spatial voxel grid having coordinates (X, Y, Z)>And->For focal length->And->And the coordinates of the center point of the image in the virtual screen.
6. The three-dimensional model output method according to claim 5, wherein before the outputting of the target image, the three-dimensional model output method further comprises:
and converting the coordinates of each pixel on the target image into a two-dimensional coordinate system.
7. The three-dimensional model output method according to claim 1, wherein after the outputting of the target image, the three-dimensional model output method further comprises:
acquiring a second initial image;
superposing the second initial image and the target image to obtain a new target image;
and outputting the new target image.
8. A three-dimensional model output device, comprising:
a first acquisition module for acquiring a plurality of first initial images photographed by cameras of different camera positions;
the construction module is used for carrying out three-dimensional reconstruction according to the plurality of first initial images to obtain a three-dimensional model;
the first conversion module is used for projecting the three-dimensional model so as to obtain a target image on a virtual screen;
and the first output module is used for outputting the target image.
9. A three-dimensional model output apparatus, comprising: the device comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete communication with each other through the communication bus;
the memory is used for storing at least one program, and the program causes the processor to execute the operation of the three-dimensional model output method according to any one of claims 1 to 7.
10. A computer readable storage medium, wherein at least one program is stored in the storage medium, which when run on a three-dimensional model output device causes the three-dimensional model output device to perform the operations of the three-dimensional model output method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311290411.4A CN117036444A (en) | 2023-10-08 | 2023-10-08 | Three-dimensional model output method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311290411.4A CN117036444A (en) | 2023-10-08 | 2023-10-08 | Three-dimensional model output method, device, equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117036444A true CN117036444A (en) | 2023-11-10 |
Family
ID=88632217
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311290411.4A Pending CN117036444A (en) | 2023-10-08 | 2023-10-08 | Three-dimensional model output method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117036444A (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106228594A (en) * | 2016-07-18 | 2016-12-14 | 中国人民解放军理工大学 | Typhoon model cloud cartoon display method based on surface subdivision |
| CN107564089A (en) * | 2017-08-10 | 2018-01-09 | 腾讯科技（深圳）有限公司 | Three dimensional image processing method, device, storage medium and computer equipment |
| CN108600607A (en) * | 2018-03-13 | 2018-09-28 | 上海网罗电子科技有限公司 | A kind of fire-fighting panoramic information methods of exhibiting based on unmanned plane |
| CN112365673A (en) * | 2020-11-12 | 2021-02-12 | 光谷技术股份公司 | Forest fire monitoring system and method |
| CN112435427A (en) * | 2020-11-12 | 2021-03-02 | 光谷技术股份公司 | Forest fire monitoring system and method |
| CN112991458A (en) * | 2021-03-09 | 2021-06-18 | 武汉大学 | Rapid three-dimensional modeling method and system based on voxels |
| CN113808277A (en) * | 2021-11-05 | 2021-12-17 | 腾讯科技（深圳）有限公司 | Image processing method and related device |
| CN114047823A (en) * | 2021-11-26 | 2022-02-15 | 北京房江湖科技有限公司 | Three-dimensional model display method, computer-readable storage medium and electronic device |
| CN116018619A (en) * | 2020-07-01 | 2023-04-25 | 索尼集团公司 | Information processing device, information processing method, and program |
-
2023
- 2023-10-08 CN CN202311290411.4A patent/CN117036444A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106228594A (en) * | 2016-07-18 | 2016-12-14 | 中国人民解放军理工大学 | Typhoon model cloud cartoon display method based on surface subdivision |
| CN107564089A (en) * | 2017-08-10 | 2018-01-09 | 腾讯科技（深圳）有限公司 | Three dimensional image processing method, device, storage medium and computer equipment |
| CN108600607A (en) * | 2018-03-13 | 2018-09-28 | 上海网罗电子科技有限公司 | A kind of fire-fighting panoramic information methods of exhibiting based on unmanned plane |
| CN116018619A (en) * | 2020-07-01 | 2023-04-25 | 索尼集团公司 | Information processing device, information processing method, and program |
| CN112365673A (en) * | 2020-11-12 | 2021-02-12 | 光谷技术股份公司 | Forest fire monitoring system and method |
| CN112435427A (en) * | 2020-11-12 | 2021-03-02 | 光谷技术股份公司 | Forest fire monitoring system and method |
| CN112991458A (en) * | 2021-03-09 | 2021-06-18 | 武汉大学 | Rapid three-dimensional modeling method and system based on voxels |
| CN113808277A (en) * | 2021-11-05 | 2021-12-17 | 腾讯科技（深圳）有限公司 | Image processing method and related device |
| CN114047823A (en) * | 2021-11-26 | 2022-02-15 | 北京房江湖科技有限公司 | Three-dimensional model display method, computer-readable storage medium and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021174939A1 (en) | Facial image acquisition method and system | |
| CN108665536B (en) | Three-dimensional and live-action data visualization method and device and computer readable storage medium | |
| US11893701B2 (en) | Method for simulating natural perception in virtual and augmented reality scenes | |
| CN109697688A (en) | A kind of method and apparatus for image procossing | |
| CN112639846A (en) | Method and device for training deep learning model | |
| CN108509887A (en) | A kind of acquisition ambient lighting information approach, device and electronic equipment | |
| JP2006053694A (en) | Space simulator, space simulation method, space simulation program and recording medium | |
| CN106558017B (en) | Spherical display image processing method and system | |
| AU2019226134B2 (en) | Environment map hole-filling | |
| CN112651881B (en) | Image synthesizing method, apparatus, device, storage medium, and program product | |
| US11276150B2 (en) | Environment map generation and hole filling | |
| CN115641401A (en) | Construction method and related device of three-dimensional live-action model | |
| WO2022100419A1 (en) | Image processing method and related device | |
| CN114143528A (en) | Multi-video stream fusion method, electronic device and storage medium | |
| WO2017113729A1 (en) | 360-degree image loading method and loading module, and mobile terminal | |
| CN114549289A (en) | Image processing method, image processing device, electronic equipment and computer storage medium | |
| CN113724391A (en) | Three-dimensional model construction method and device, electronic equipment and computer readable medium | |
| WO2022166868A1 (en) | Walkthrough view generation method, apparatus and device, and storage medium | |
| CN109427089B (en) | Mixed reality object presentation based on ambient lighting conditions | |
| CN109446945A (en) | Threedimensional model treating method and apparatus, electronic equipment, computer readable storage medium | |
| CN109166176B (en) | Three-dimensional face image generation method and device | |
| CN117036444A (en) | Three-dimensional model output method, device, equipment and computer readable storage medium | |
| CN114520903A (en) | Rendering display method, device, storage medium and computer program product | |
| CN114066731A (en) | Method and device for generating panorama, electronic equipment and storage medium | |
| CN114241127A (en) | Panoramic image generation method and device, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |