CN117197319B - Image generation method, device, electronic equipment and storage medium - Google Patents
Image generation method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117197319B CN117197319B CN202311472485.XA CN202311472485A CN117197319B CN 117197319 B CN117197319 B CN 117197319B CN 202311472485 A CN202311472485 A CN 202311472485A CN 117197319 B CN117197319 B CN 117197319B
- Authority
- CN
- China
- Prior art keywords
- map
- azimuth
- cube
- adjusted
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 74
- 230000000903 blocking effect Effects 0.000 claims description 12
- 230000004927 fusion Effects 0.000 claims description 7
- 239000000758 substrate Substances 0.000 claims 2
- 238000005516 engineering process Methods 0.000 abstract description 16
- 238000013473 artificial intelligence Methods 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 16
- 238000004590 computer program Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 9
- 238000009877 rendering Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 238000012549 training Methods 0.000 description 7
- 238000004891 communication Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000010801 machine learning Methods 0.000 description 5
- 238000010276 construction Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000005855 radiation Effects 0.000 description 4
- 239000000969 carrier Substances 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
Abstract
The application discloses an image generation method, an image generation device, electronic equipment and a storage medium. The embodiment of the application relates to the technical fields of artificial intelligence, cloud technology and the like. The method comprises the following steps: carrying out three-dimensional reconstruction on a sample shooting image obtained by shooting a target object according to a plurality of sample viewpoints and each sample viewpoint to obtain a three-dimensional model of the target object; observing the three-dimensional model to obtain a target observation result corresponding to the target object; the target observation result comprises a plurality of cube maps, texture maps and depth maps corresponding to the texture maps; fusing the cube maps to obtain a panoramic image showing a three-dimensional model; and generating an image according to the target viewpoint, the texture map, the depth map and the panorama to obtain a target image when the target object is watched from the target viewpoint. The efficiency of generating the target image according to the application method is high.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image generating method, an image generating device, an electronic device, and a storage medium.
Background
The pseudo 3D indoor effect is to lay a customized material on the surface of the window, the material uses a depth map, a color map and a viewing viewpoint of a player as input, a rendering result can be calculated in real time along with the viewpoint change of the player, and a perspective view of the room at the corresponding viewpoint is presented. Such a solution enables a player to view the 3D effects of a room, but does not actually enter the room for play and interaction.
At present, a point cloud model of a target room can be constructed through shooting images shot by a plurality of viewpoints on the target room, and then the point cloud model of the target room is subjected to three-dimensional rendering according to the input target viewing angle, so that a pseudo 3D indoor effect diagram for viewing the target room from the target viewing angle is obtained. However, when the method is adopted to carry out three-dimensional rendering on the point cloud model of the target room, the data operand is large, and the efficiency of generating the pseudo 3D indoor effect graph is low.
Disclosure of Invention
In view of this, an embodiment of the present application provides an image generating method, an image generating device, an electronic device, and a storage medium.
In a first aspect, an embodiment of the present application provides an image generating method, including: carrying out three-dimensional reconstruction on a sample shooting image obtained by shooting a target object according to a plurality of sample viewpoints and each sample viewpoint to obtain a three-dimensional model of the target object; observing the three-dimensional model to obtain a target observation result corresponding to the target object; the target observation result comprises a plurality of cube maps, texture maps and depth maps corresponding to the texture maps; fusing the cube maps to obtain a panoramic image showing a three-dimensional model; and generating an image according to the target viewpoint, the texture map, the depth map and the panorama to obtain a target image when the target object is watched from the target viewpoint.
In a second aspect, an embodiment of the present application provides an image generating apparatus, including: the acquisition module is used for carrying out three-dimensional reconstruction on a sample shooting image obtained by shooting a target object according to a plurality of sample viewpoints and each sample viewpoint to obtain a three-dimensional model of the target object; the determining module is used for observing the three-dimensional model to obtain a target observation result corresponding to the target object; the target observation result comprises a plurality of cube maps, texture maps and depth maps corresponding to the texture maps; the fusion module is used for fusing the plurality of cube maps to obtain a panoramic image showing the three-dimensional model; and the generation module is used for generating images according to the target viewpoint, the texture map, the depth map and the panorama to obtain a target image when the target object is watched from the target viewpoint.
Optionally, the texture map comprises a plurality of azimuthal texture maps observed from a plurality of preset viewpoints; the depth map comprises azimuth depth maps respectively corresponding to the azimuth texture maps; a determination module for determining a cube surrounding the three-dimensional model; determining a plurality of preset viewpoints according to the positions of the cubes; observing the surface of the cube from a plurality of preset viewpoints respectively to obtain a plurality of azimuth texture maps and azimuth depth maps corresponding to each azimuth texture map; and observing the space surrounded by the cube along the direction perpendicular to each surface of the cube by taking the center of the cube as an observation position to obtain a plurality of cube maps.
Optionally, the target observation result further includes an observation parameter corresponding to the depth map; the observation parameters of the depth map are used for indicating the distance between the observation surface corresponding to the depth map and the preset viewpoint corresponding to the depth map; the observation surface is a surface on a cube surrounding the three-dimensional model; the device also comprises an adjusting module, a display module and a display module, wherein the adjusting module is used for adjusting the observation parameters corresponding to the depth map to obtain an adjusted cube; determining an adjusted depth map and an adjusted texture map according to the adjusted cube; correspondingly, the generating module is further used for generating an image according to the target viewpoint, the adjusted texture map, the adjusted depth map and the panorama, so as to obtain a target image when the target object is watched from the target viewpoint.
Optionally, the texture map comprises azimuth texture maps respectively observed from a plurality of preset viewpoints; the depth map comprises azimuth depth maps respectively corresponding to the azimuth texture maps; the adjusted texture map comprises an adjusted azimuth texture map corresponding to each azimuth texture map; the adjusted depth map comprises an adjusted azimuth depth map corresponding to each azimuth depth map; the adjusting module is also used for determining a first azimuth texture map in which the target object is blocked by the blocking element and pixel position information of the blocking element in the first azimuth texture map in the plurality of azimuth texture maps; determining depth information of the shielding element in a first azimuth depth map corresponding to the first azimuth texture map according to pixel position information of the shielding element in the first azimuth texture map; adjusting the observation parameters of the first azimuth depth map according to the depth information of the shielding element to obtain an adjusted cube; re-observing the surface of the adjusted cube to obtain an adjusted azimuth depth map and an azimuth texture map; the area of the pixel area occupied by the shielding element in the adjusted first azimuth depth map is smaller than that of the pixel area occupied in the first azimuth depth map.
Optionally, the texture map comprises azimuth texture maps respectively observed from a plurality of preset viewpoints; the depth map comprises azimuth depth maps respectively corresponding to the azimuth texture maps; the adjusted texture map comprises an adjusted azimuth texture map corresponding to each azimuth texture map; the adjusted depth map comprises an adjusted azimuth depth map corresponding to each azimuth depth map; the adjusting module is further used for determining a second azimuth depth map comprising the target class area and pixel position information of the target class area in the second azimuth depth map from the plurality of azimuth depth maps; the target class area is a pixel area presenting other observation surfaces in the second azimuth depth map; the other observation surfaces are observation surfaces corresponding to other azimuth texture maps except the second azimuth texture map in the plurality of azimuth texture maps; the second azimuth texture map is an azimuth texture map corresponding to the second azimuth depth map; determining depth information of the target class region in the second azimuth depth map according to pixel position information of the target class region in the second azimuth depth map; adjusting the observation parameters of a third azimuth depth map in the azimuth depth maps according to the depth information of the target class region to obtain an adjusted cube; re-observing the surface of the adjusted cube to obtain an adjusted azimuth depth map and an azimuth texture map; the area of the pixel area occupied by the target class area in the adjusted second azimuth depth map is smaller than that of the pixel area occupied by the target class area in the second azimuth depth map.
Optionally, the adjusting module is further configured to adjust an observation parameter of the depth map to a first observation parameter in response to a triggering operation for the first adjusting control, so as to obtain an adjusted cube; the first observation parameter is input in a triggering way; re-observing the surface of the adjusted cube to obtain an adjusted azimuth depth map and an azimuth texture map; and displaying the adjusted depth map and the adjusted texture map in the first image display area.
Optionally, the target observation result further comprises observation parameters corresponding to each cube map; the observation parameters of the cube map are used for indicating the focal length of the view point corresponding to the cube map; the adjusting module is also used for adjusting the observation parameters of the target cube map in the plurality of cube maps to obtain an adjusted target cube map; correspondingly, the fusion module is further used for fusing the adjusted target cube map and other cube maps except the target cube map in the plurality of cube maps to obtain a panoramic image presenting the three-dimensional model.
Optionally, the target cube map is a cube map having a sharpness less than a preset sharpness threshold; the adjusting module is also used for adjusting the observation parameters of the target cube map according to the definition of the target cube map to obtain an adjusted target cube map; the sharpness of the target cube map is lower than the sharpness of the adjusted target cube map.
Optionally, the adjustment module is further configured to display a plurality of cube maps in a second image display area of the display interface; the second display interface also comprises a second image display area, and the second image display area comprises a second adjustment control corresponding to each cube map; responding to the triggering operation of a second adjustment control corresponding to the target cube map, and adjusting the observation parameters of the target cube map to second observation parameters to obtain an adjusted target cube map; the second observation parameter is input in a triggering way; and displaying the adjusted target cube map in a second image display area of the display interface.
Optionally, the texture map comprises a plurality of azimuthal texture maps observed from a plurality of preset viewpoints; the depth map comprises azimuth depth maps respectively corresponding to the azimuth texture maps; the generation module is also used for combining the plurality of azimuth depth maps into a combined depth map of multiple channels; the number of channels of the combined depth map is the same as the number of azimuth depth maps; splicing a fourth azimuth texture map and a fifth azimuth texture map in the azimuth texture maps to obtain a spliced texture map; the fourth azimuth texture map is an azimuth texture map observed from the first preset viewpoint; the fifth azimuth texture map is an azimuth texture map observed from a second preset viewpoint; and generating a target image when the target object is watched from the target viewpoint according to the target viewpoint, the combined depth map, the spliced texture map and the panorama.
Optionally, the apparatus further comprises a sample acquisition module for displaying a task creation interface; the task creation interface includes a selection control; and responding to the selection operation of the selection control, acquiring a plurality of selected sample viewpoints and sample shooting images obtained by shooting the target object at each sample viewpoint.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory; the memory has stored thereon computer readable instructions which, when executed by the processor, implement the method described above.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium having computer-readable instructions stored thereon, which when executed by a processor, implement the above-described method.
In a fifth aspect, embodiments of the present application provide a computer program product or computer program comprising computer instructions which, when executed by a processor, implement the above-described method.
In the image generating method, the device, the electronic equipment and the storage medium provided by the embodiment of the application, the three-dimensional model of the target object is directly observed to determine a plurality of cube maps, texture maps and depth maps corresponding to the texture maps corresponding to the target object, fusion is carried out according to the cube maps to obtain a panorama representing the three-dimensional model, the texture maps, the depth maps and the panorama are used as carriers of information, rendering is carried out according to a target viewpoint to obtain the target image when the target object is watched from the target viewpoint, three-dimensional rendering is not needed to be carried out on the three-dimensional model to obtain the target image, and the rendering efficiency of the image is higher than that of the three-dimensional model, so that the efficiency of generating the target image according to the application method is higher.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 shows a schematic diagram of an application scenario applicable to an embodiment of the present application;
FIG. 2 illustrates a flow chart of an image generation method according to one embodiment of the present application;
FIG. 3 illustrates a schematic diagram of a task creation interface in an embodiment of the present application;
FIG. 4 is a schematic diagram of a file selection interface according to an embodiment of the present application;
FIG. 5 illustrates a schematic diagram of a cube map in an embodiment of the present application;
FIG. 6 illustrates a panorama corresponding to the cube map illustrated in FIG. 5;
FIG. 7 is a flowchart showing steps in one embodiment after step S120 of the corresponding embodiment of FIG. 2;
FIG. 8 is a flow chart showing steps in an embodiment prior to step S210 of the corresponding embodiment of FIG. 7;
FIG. 9 is a schematic diagram of a first image presentation area according to an embodiment of the present application;
FIG. 10 is a schematic view of the first image presentation area of FIG. 9 after adjustment of the depth map;
FIG. 11 is a schematic view of the first image presentation area of FIG. 10 after adjustment of the depth map;
FIG. 12 is a schematic view of the first image presentation area of FIG. 11 after adjustment of the depth map;
FIG. 13 shows a flow chart of steps in an embodiment after step S120 of the corresponding embodiment of FIG. 2;
FIG. 14 is a flow chart showing steps in an embodiment prior to step S410 of the corresponding embodiment of FIG. 13;
FIG. 15 is a schematic view of a second image presentation area in an embodiment of the present application;
FIG. 16 is a schematic view of a second image display area of the cube map of FIG. 15 after adjustment;
FIG. 17 is a schematic diagram of a target image in an embodiment of the present application;
FIG. 18 is a schematic diagram of a process for acquiring a target image according to an embodiment of the present application;
FIG. 19 shows a block diagram of an image generation apparatus according to one embodiment of the present application;
fig. 20 shows a block diagram of a configuration of an electronic device for executing an image generating method according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
In the following description, the terms "first", "second", and the like are merely used to distinguish similar objects and do not represent a particular ordering of the objects, it being understood that the "first", "second", and the like may be interchanged with one another, if permitted, to enable embodiments of the application described herein to be practiced otherwise than as illustrated or described herein.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of the present application only and is not intended to be limiting of the present application.
It should be noted that: references herein to "a plurality" means two or more. "and/or" describes an association relationship of an association object, meaning that there may be three relationships, e.g., a and/or B may represent: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
The application discloses an image generation method, an image generation device, electronic equipment and a storage medium, and relates to an artificial intelligence technology.
Artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use the knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.
The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
Machine Learning (ML) is a multi-domain interdisciplinary, involving multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, induction learning, and the like.
As shown in fig. 2, an application scenario applicable to the embodiment of the present application includes a terminal 20 and a server 10, where the terminal 20 and the server 10 are connected through a wired network or a wireless network. The terminal 20 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart home appliance, a vehicle-mounted terminal, an aircraft, a wearable device terminal, a virtual reality device, and other terminal devices capable of page presentation, or other applications (e.g., instant messaging applications, shopping applications, search applications, game applications, forum applications, map traffic applications, etc.) capable of invoking page presentation applications.
The server 10 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs (Content Delivery Network, content delivery networks), basic cloud computing services such as big data and artificial intelligent platforms, and the like. The server 10 may be used to provide services for applications running at the terminal 20.
The terminal 20 may send a plurality of sample viewpoints and a sample shooting image obtained by shooting the target object at each sample viewpoint to the server 10, and the server 10 may perform three-dimensional reconstruction according to the plurality of sample viewpoints and the sample shooting image obtained by shooting the target object at each sample viewpoint to obtain a three-dimensional model of the target object; determining a target observation result corresponding to the target object according to the three-dimensional model; the target observation result comprises a plurality of cube maps, texture maps and depth maps corresponding to the texture maps; fusing the cube maps to obtain a panoramic image showing a three-dimensional model; and generating images according to the target viewpoint, the texture map, the depth map and the panorama to obtain a target image when the three-dimensional model is watched from the target viewpoint, and finally, returning the target image of the server 10 to the terminal 20.
In another embodiment, the server 10 may obtain a plurality of sample viewpoints and a sample photographed image obtained by photographing the target object at each sample viewpoint, and then the server 10 may perform three-dimensional reconstruction according to the plurality of sample viewpoints and the sample photographed image obtained by photographing the target object at each sample viewpoint to obtain a three-dimensional model of the target object; determining a target observation result corresponding to the target object according to the three-dimensional model; the target observation result comprises a plurality of cube maps, texture maps and depth maps corresponding to the texture maps; and fusing the plurality of cube maps to obtain a panoramic image presenting the three-dimensional model. The terminal 20 may send the obtained target viewpoint to the server 10, and the server 10 generates an image according to the target viewpoint, the texture map, the depth map, and the panorama map, so as to obtain a target image when the three-dimensional model is viewed from the target viewpoint. The server 10 then returns the target image to the terminal.
In yet another embodiment, the terminal 20 may be configured to perform the methods of the present application to obtain a target image.
It may be appreciated that after the server 10 constructs the three-dimensional model of the target object, the three-dimensional model of the target object may be stored in the distributed cloud storage system, and the terminal 20 obtains the three-dimensional model of the target object from the distributed cloud storage system, and then determines the target image according to the three-dimensional model of the target object and the target viewpoint.
For convenience of description, in the following embodiments, an example in which an image generating method is executed by an electronic apparatus will be described.
Referring to fig. 2, fig. 2 is a flowchart of an image generating method according to an embodiment of the present application, where the method may be applied to an electronic device, and the electronic device may be the terminal 20 or the server 10 in fig. 2, and the method includes:
s110, carrying out three-dimensional reconstruction on a sample shooting image obtained by shooting the target object according to a plurality of sample viewpoints and each sample viewpoint to obtain a three-dimensional model of the target object.
In this embodiment, the target object may be an entity to be subjected to three-dimensional reconstruction, and the target object may be a real entity of a real environment, for example, the target object may be a room, a whole building, or an outdoor local area, etc.; the target object may also be a virtual entity of a virtual environment, e.g. the target object may be a virtual room or a virtual office building, etc.
The viewpoint refers to a pose for observing the target object, and the viewpoint may include an observation position to observe an angle. And shooting the target object from the observation position included in each sample viewpoint at an observation angle to obtain respective sample shooting images of each sample viewpoint.
The sample viewpoint refers to a viewpoint from which a sample captured image is acquired. The sample viewpoint may be determined based on the area occupied by the target object and/or object attributes of the target object, where the object attributes may include indoor attributes or outdoor attributes. For example, the larger the area occupied by the target object, the more sample viewpoints; the number of sample viewpoints corresponding to the target object whose object attribute is an indoor attribute is higher than the number of sample viewpoints of the target object whose object attribute is an outdoor attribute.
In order to enable accurate three-dimensional reconstruction of the target object, the number of sample viewpoints may be set to be large so that the resulting sample captured images at the plurality of sample viewpoints may include all of the target object.
When the object class of the target object is indoor, the three-dimensional reconstruction of the target object may be called indoor reconstruction, where the indoor reconstruction refers to that a group of photographs (photographed images corresponding to a plurality of sample viewpoints) are photographed according to a surrounding static indoor scene, and a three-dimensional effect (such as a three-dimensional model or a three-dimensional animation) of the indoor scene is recovered from the group of two-dimensional photographs, which is called indoor reconstruction.
In some embodiments, the parameter-initialized NeRF (Neural Radiance Fields, neural radiation field) may be trained through a plurality of sample viewpoints and a sample photographed image obtained by photographing the target object at each sample viewpoint, so as to obtain a trained NeRF, and the three-dimensional radiation field obtained according to the trained NeRF is used as the three-dimensional model of the target object. The three-dimensional radiation field refers to a three-dimensional observation result obtained by observing a certain space at a certain point in the three-dimensional space.
NeRF is a computer vision technique for generating high quality three-dimensional models. The method utilizes a deep learning technology to extract geometric shape and texture information of an object from images at a plurality of viewpoints, and then uses the information to generate a continuous three-dimensional radiation field, so that a highly realistic three-dimensional model can be presented at any angle and distance. NeRF technology has wide application prospect in the fields of computer graphics, virtual reality, augmented reality and the like.
In some embodiments, before S110, the method may further include: displaying a task creation interface; the task creation interface includes a selection control; and responding to the selection operation of the selection control, acquiring a plurality of selected sample viewpoints and sample shooting images obtained by shooting the target object at each sample viewpoint.
The electronic device may be provided with three-dimensional reconstruction software, and after opening the three-dimensional reconstruction software, a task creation interface may be displayed, which may include a selection control.
As shown in fig. 3, the task creation interface 30 may expose a state of a task, which may refer to a three-dimensional model building task. The task id refers to the id (identification) of the three-dimensional model construction task, the progress can refer to the task progress of the three-dimensional model construction task, and when the progress is 100%, the three-dimensional model construction task is indicated to be completed. The task creation interface 30 may also include an operation control, where the operation control includes an operation control for each task, where the operation control for each task includes a view log control and a view detail control, where a user clicks on the view log control to view a log of a three-dimensional reconstruction process of a corresponding three-dimensional model building task, and where a user clicks on the view log control to view specific information of the corresponding three-dimensional model building task (e.g., cube maps and depth maps corresponding to the three-dimensional model, etc.).
The task creation interface 30 further includes a selection control 31, where the selection control 31 may include a selection file control and a creation task control, and may click on the creation task control to create a new task, or may click on the selection file control to select an image required by the new task.
After clicking the file selection control, as shown in fig. 4, the file selection interface 32 may be output, and the user may select files under different directories in the file selection interface 32 as basic data for constructing the three-dimensional model, or may input a file name in the file name input control 321 in the file selection interface 32 to directly obtain the basic data for constructing the three-dimensional model. As shown in fig. 4, the file selection interface 32 may be displayed above the task creation interface 30.
And S120, observing the three-dimensional model to obtain a target observation result corresponding to the target object.
The target observation result comprises a plurality of cube maps, texture maps and depth maps corresponding to the texture maps. The texture map comprises a plurality of azimuth texture maps observed from a plurality of preset viewpoints; the depth map comprises azimuth depth maps respectively corresponding to the azimuth texture maps; the cube map may be a color image.
After the three-dimensional model is obtained, observation operation can be carried out on the three-dimensional model, and a plurality of cube maps, texture maps and depth maps corresponding to the texture maps are obtained. Wherein, the observing operation may include: determining a cube surrounding the three-dimensional model; determining a plurality of preset viewpoints according to the positions of the cubes; observing the surface of the cube from a plurality of preset viewpoints respectively to obtain a plurality of azimuth texture maps and azimuth depth maps corresponding to each azimuth texture map; and observing the space surrounded by the cube along the direction perpendicular to each surface of the cube by taking the center of the cube as an observation position to obtain a plurality of cube maps.
In this embodiment, 6 faces including the maximum number of vertices (intersections between edge lines in the three-dimensional model, edge lines refer to intersections of edge faces (also called outermost faces) in the three-dimensional model) may be obtained from the three-dimensional model by a RANSAC (Random Sample Consensus ) algorithm, and a cube formed by the 6 faces is used as a cube surrounding the three-dimensional model, and each face of the cube is used as an observation face. After obtaining a cube comprising a three-dimensional model, a plurality of preset viewpoints are determined based on the position of the cube. The preset viewpoints may be located on the surface of the cube or may be located inside the cube, and it is worth mentioning that the preset viewpoints are viewpoints from which a three-dimensional model surrounded by the cube can be observed.
The determined cube is used to define the scope of the observation, and since the cube encloses the three-dimensional model of the target object, a plurality of texture maps obtained by observing the space enclosed by the cube, or a three-dimensional model of the target object can be presented in the cube map. It should be noted that, since other elements may be presented in the space where the target object is located, the multiple sample captured images acquired by the target object may be presented with other elements (or other objects) in the space where the target object is located, and the three-dimensional model of the target object is obtained by three-dimensional reconstruction according to the multiple sample captured images acquired by the target object, other objects unrelated to the target object may be presented in the model obtained by three-dimensional reconstruction. In the application, in order to avoid that other objects irrelevant to the target object are excessively presented in the generated target image, the spatial range for observation is limited by the cube, so that even if the model obtained by reconstruction comprises the model of other objects irrelevant to the target object, since only the space limited by the cube is observed, the content presented by the depth map corresponding to the observed cube map, texture map and texture map is information in the space limited by the cube, and information in other spatial ranges outside the cube cannot be presented.
One face in the cube may be a positive observation face, and based on this positive observation face, a rear observation face facing the positive observation face, a left observation face to the left of the positive observation face, a right observation face to the right of the positive observation face, a bottom observation face below the positive observation face, and a top observation face above the positive observation face may be determined. For each observation plane, a preset viewpoint may be determined: the top view point corresponding to the top observation surface, the bottom view point corresponding to the bottom observation surface, the right view point corresponding to the right side observation, the left view point corresponding to the left observation surface, the front view point corresponding to the front observation surface, and the rear view point corresponding to the rear observation surface, and in addition, the front 45-degree view point can be determined as a preset view point.
The observation position of the forward viewpoint is positioned in the center of the forward observation surface, and the observation direction of the forward viewpoint is perpendicular to the direction of the forward observation surface; based on the forward view point, the face on the cube being observed is another face (i.e., the rear viewing face) that is parallel to the viewing face where the forward view point is located. The observation position of the rear view point is the center of the rear observation surface, and the observation direction of the rear view point is the direction perpendicular to the rear observation surface; the observation position of the left viewpoint is the center of the left observation surface, and the observation direction of the left viewpoint is the direction perpendicular to the left observation surface; the observation position of the right viewpoint is the center of the right observation surface, and the observation direction of the right viewpoint is the direction perpendicular to the right observation surface; the observation position of the top viewpoint is the center of the top observation surface, and the observation direction of the top viewpoint is the direction perpendicular to the top observation surface; the observation position of the bottom viewpoint is the center of the bottom observation surface, and the observation direction of the bottom viewpoint is the direction perpendicular to the bottom observation surface; the observation position of the positive 45-degree viewpoint is the midpoint of the top edge of the positive observation surface, and the observation direction of the positive 45-degree viewpoint is the direction perpendicular to the center of the cube.
After determining the plurality of preset viewpoints, a virtual camera may be set at an observation position included by each preset viewpoint, and the three-dimensional model surrounded by the cube is observed by the virtual camera according to an observation direction of each preset viewpoint to obtain an azimuth texture map and an azimuth depth map corresponding to each preset viewpoint, where the azimuth texture map and the azimuth depth map correspond to each other, and the azimuth texture map and the azimuth depth map may refer to one-to-one correspondence between a pixel point in the azimuth texture map and a pixel point in the azimuth depth map, that is, the two pixel points represent the same element, information of the pixel in the azimuth texture map is color information of the represented element, and information of the pixel in the azimuth depth is a depth value of the represented element. In other words, the depth value of each depth pixel point in the depth map corresponding to the preset viewpoint is the distance between the entity part represented by the color pixel point corresponding to the depth pixel point and the preset viewpoint in the texture map corresponding to the preset viewpoint.
For example, the target object is a room F1, the azimuth depth map D1 and the azimuth texture map W1 are an azimuth depth map and an azimuth texture map under the same preset viewpoint sd1, the pixel points in the azimuth depth map D1 and the pixel points in the azimuth texture map W1 are in one-to-one correspondence, and the pixel point dx1 in the azimuth depth map D1 and the pixel point wx1 in the azimuth texture map W1 correspond, so that the depth value of the pixel point dx1 is the distance between the solid parts g1 and sd1 in the room F1 represented by wx 1.
And aiming at any one preset viewpoint, observing the three-dimensional model surrounded by the cube according to the preset viewpoint to obtain an azimuth texture map and an azimuth depth map corresponding to the preset viewpoint, wherein an observation surface observed by the preset viewpoint is used as an observation surface corresponding to the azimuth texture map and the azimuth depth map observed by the preset viewpoint, namely, the observation surfaces corresponding to the azimuth texture map and the azimuth depth map under the same preset viewpoint are the same. For example, if the observation plane observed from the forward viewpoint is the rear observation plane, the rear observation plane is the observation plane of the azimuth texture map and the azimuth depth map corresponding to the forward viewpoint.
Meanwhile, after the cube including the three-dimensional model is obtained, each observation surface can be observed from the center of the cube (the direction perpendicular to the observation surface), and the image observed by each observation surface can be obtained as a cube map corresponding to each observation surface.
And S130, fusing the plurality of cube maps to obtain a panoramic image presenting the three-dimensional model.
After obtaining the cube maps, the texture maps and the depth maps corresponding to the texture maps, the cube maps can be continuously fused to obtain a panoramic map presenting a three-dimensional model.
In this embodiment, a panorama of the three-dimensional model may be generated from the plurality of cube maps by cube2 sphere. For example, the obtained cube map is shown in fig. 5, and the panorama obtained by the open source algorithm cube2sphere according to the cube map shown in fig. 5 is shown in fig. 6.
And S140, generating an image according to the target viewpoint, the texture map, the depth map and the panorama, and obtaining a target image when the target object is watched from the target viewpoint.
The user can set a target viewpoint according to the requirement, and then the electronic device generates an image according to the target viewpoint, the texture map, the depth map and the panorama by using an illusion engine (for example, UE 5) to obtain a target image when the target object is watched from the target viewpoint.
Optionally, in some embodiments, S140 may include: combining the multiple azimuth depth maps into a combined depth map of multiple channels; the number of channels of the combined depth map is the same as the number of azimuth depth maps; splicing a fourth azimuth texture map and a fifth azimuth texture map in the azimuth texture maps to obtain a spliced texture map; the fourth azimuth texture map is an azimuth texture map observed from the first preset viewpoint; the fifth azimuth texture map is an azimuth texture map observed from a second preset viewpoint; and generating a target image when the target object is watched from the target viewpoint according to the target viewpoint, the combined depth map, the spliced texture map and the panorama.
As described above, when the preset views include a forward view, a backward view, a left view, a right view, a backward view, and a forward 45 degree view, the forward 45 degree view, the backward view, and the top view may be selected from the seven preset views as target preset views, and the azimuth depth maps observed by the target preset views are combined to obtain a combined depth map, where the number of channels of the combined depth map is the same as the number of target preset views (4 channels), the combined depth map may be in RGBA format, and the image under each channel of the RGBA combined depth map is the depth map observed by the target preset view.
Wherein the first preset viewpoint may be a positive viewpoint and the second preset viewpoint may be a positive 45 degree viewpoint. The fourth azimuth texture map and the fifth azimuth texture map may be stitched in a horizontal direction: the fourth azimuth texture map is spliced on the right side of the fifth azimuth texture map, or the fourth azimuth texture map is spliced on the left side of the fifth azimuth texture map.
After the merged depth map and the spliced texture map are obtained, an illusion engine (for example, UE 5) can generate an image according to the target viewpoint, the merged depth map, the spliced texture map and the panorama, so as to obtain a target image when the target object is watched from the target viewpoint.
In some embodiments, when the method is used in a game scene, a three-dimensional reconstruction can be performed on a displayed target object to obtain a three-dimensional model, the three-dimensional model is set in a certain area of the game scene, when a character in the game is in the vicinity of the three-dimensional model, a viewpoint of the character can be determined as a target viewpoint according to the pose of the character, and then image generation is performed according to the target viewpoint, a texture map, a depth map and a panorama map to obtain a target image when the character views the target object from the target viewpoint.
In this embodiment, the three-dimensional model of the target object is directly observed to determine a plurality of cube maps, texture maps and depth maps corresponding to the texture maps corresponding to the target object, and the three-dimensional maps are fused according to the plurality of cube maps to obtain a panorama representing the three-dimensional model, the texture maps, the depth maps and the panorama are used as carriers of information, and the three-dimensional model is not required to be stereoscopically rendered to obtain the target image when the target object is viewed from the target viewpoint, and the rendering efficiency of the image is higher than that of the three-dimensional model, and the rendering speed block is higher, so that the efficiency of generating the target image according to the application method is higher.
And secondly, three pictures of a texture map, a depth map and a panoramic map are used as information carriers obtained by reconstruction, and picture information can be seamlessly connected with a illusion engine and can be directly used in the illusion engine, so that efficient acquisition of a target image is realized.
In addition, compared with manual generation of the target image, the method can produce the target image with high quality and high reduction degree. The method can also be used for reconstructing the target object in the virtual environment, and the applicability and the universality of reconstruction are improved.
In some embodiments, as shown in fig. 7, after S120, the method may further include:
s210, adjusting observation parameters corresponding to the depth map to obtain an adjusted cube; and determining an adjusted depth map and an adjusted texture map according to the adjusted cube.
The target observation result also comprises observation parameters corresponding to the depth map; the observation parameters of the depth map are used for indicating the distance between the observation surface corresponding to the depth map and the preset viewpoint corresponding to the depth map. The distance between the observation surface corresponding to the depth map and the preset viewpoint corresponding to the depth map is inversely related to the observation parameter corresponding to the depth map: the larger the observation parameters corresponding to the depth map, the smaller the distance between the observation surface corresponding to the depth map and the preset viewpoint corresponding to the depth map, and conversely, the smaller the observation parameters corresponding to the depth map, the larger the distance between the observation surface corresponding to the depth map and the preset viewpoint corresponding to the depth map.
As described above, the depth map may include azimuth depth maps corresponding to each of the plurality of preset viewpoints, and the respective observation parameters of all the azimuth depth maps may be adjusted, or the observation parameters of part of the azimuth depth maps may be adjusted.
The adjustment of the observation parameters of the azimuth depth map is actually equivalent to the adjustment of the distance between the preset viewpoint corresponding to the azimuth depth map and the observation surface corresponding to the azimuth depth map (normally, the position of the observation surface corresponding to the azimuth depth map is maintained, and the position of the preset viewpoint corresponding to the azimuth depth map is adjusted). For example, when the observation parameters of the azimuth depth map are adjusted down, the preset viewpoint corresponding to the azimuth depth map is actually moved away from the observation plane corresponding to the azimuth depth map, and when the observation parameters of the azimuth depth map are adjusted up, the preset viewpoint corresponding to the azimuth depth map is actually moved toward the observation plane corresponding to the azimuth depth map.
However, changing the distance between the preset viewpoint corresponding to the azimuth depth map and the observation surface corresponding to the azimuth depth map changes the distance between the observation surface where the preset viewpoint corresponding to the azimuth depth map is located and the observation surface corresponding to the azimuth depth map, and the observation surface where the preset viewpoint corresponding to the azimuth depth map is located and the observation surface corresponding to the azimuth depth map are two opposite observation surfaces in the cube, and changing the distance between the two opposite observation surfaces is equivalent to changing the size of the cube, so that adjusting the observation parameters of the depth map is actually equivalent to adjusting the size of the cube. Namely, the observation parameters corresponding to the depth map are adjusted, namely, the adjustment of the cube surrounding the three-dimensional model is realized, and the adjusted cube can be obtained.
After the adjusted cube is obtained, a plurality of new preset viewpoints can be determined according to the adjusted cube, and a plurality of azimuth depth maps and a plurality of azimuth texture maps are determined according to the plurality of new preset viewpoints and used as the adjusted azimuth depth maps and the adjusted azimuth texture maps. Because the size of the cube after adjustment changes, the observation range correspondingly changes, so that the space surrounded by the cube after adjustment is observed again, and the obtained multiple azimuth depth maps and the content presented in the multiple azimuth texture maps correspondingly change compared with the content presented in the image before adjustment.
In some embodiments, S210 may include: determining a first azimuth texture map in which a target object is blocked by an blocking element and pixel position information of the blocking element in the first azimuth texture map in the plurality of azimuth texture maps; determining depth information of the shielding element in a first azimuth depth map corresponding to the first azimuth texture map according to pixel position information of the shielding element in the first azimuth texture map; adjusting the observation parameters of the first azimuth depth map according to the depth information of the shielding element to obtain an adjusted cube; re-observing the surface of the adjusted cube to obtain an adjusted azimuth depth map and an azimuth texture map; the area of the pixel area occupied by the shielding element in the adjusted first azimuth depth map is smaller than that of the pixel area occupied in the first azimuth depth map.
The first azimuth texture map refers to an azimuth texture map, which determines that a target object is blocked by a blocking element, in a plurality of azimuth texture maps. The first azimuthal texture map may be one or more, and is not particularly limited herein.
The method comprises the steps of determining pixel position information of an occlusion element in a depth map according to pixel position information of the occlusion element in a first azimuth texture map and a corresponding relation between the first azimuth texture map and a first azimuth depth map, and determining a corresponding depth pixel point of the occlusion element in the depth map as an occlusion element depth pixel point according to the pixel position information of the occlusion element in the depth map; according to the depth value of the depth pixel point of the shielding element (namely the depth information of the shielding element), the observation parameter of the first azimuth depth map is adjusted to adjust the distance between the observation surface corresponding to the first azimuth depth map and the preset viewpoint corresponding to the first azimuth depth map, the adjustment of the cube surrounding the three-dimensional model is realized, the adjusted cube is obtained, a plurality of new preset viewpoints are determined according to the adjusted cube, a plurality of azimuth depth maps and a plurality of azimuth texture maps are determined according to the new preset viewpoints, and the azimuth depth maps and the azimuth texture maps are used as the adjusted azimuth depth maps and the adjusted azimuth texture maps.
The adjusted azimuth depth map corresponds to the azimuth depth map one by one: an azimuthal depth map observed at the forward-looking point of the cube corresponding to the adjusted azimuthal depth map observed at the forward-looking point of the adjusted cube; similarly, the azimuth depth map observed with the rear view point of the cube corresponds to the adjusted azimuth depth map observed with the rear view point of the adjusted cube; and the other is the same. That is, the adjusted azimuth depth map corresponding to the first azimuth depth map among the adjusted plurality of azimuth depth maps is the adjusted first azimuth depth map.
As described above, the distance between the observation surface corresponding to the depth map and the preset viewpoint corresponding to the depth map is inversely related to the observation parameter corresponding to the depth map, so that the larger the area of the occlusion element is, the larger the area required to be adjusted for eliminating (or reducing) the occlusion element is, the larger the observation parameter adjustment amplitude of the first azimuth depth map is, and the smaller the area of the occlusion element is, the smaller the area required to be adjusted for eliminating (or reducing) the occlusion element is. When the observation parameters of the first azimuth depth map are adjusted according to the strategy, the area of the pixel region occupied by the shielding element in the adjusted first azimuth depth map can be furthest made smaller than the area of the pixel region occupied in the first azimuth depth map, and even the shielding element can be removed from the adjusted first azimuth depth map.
It should be noted that when the shielding element exists in the first azimuth depth map, the observation parameter corresponding to the first azimuth depth map can be properly adjusted, so that the size of the cube can be reduced, the space range of observation is correspondingly reduced, and an adjusted cube is obtained, and the adjusted cube excludes all or part of the shielding element from the outside of the adjusted cube, so that the area of the shielding element in the adjusted first azimuth depth map is reduced or eliminated.
When no shielding element exists in all azimuth depth maps, the observation parameters of each azimuth depth map can be not adjusted.
In another embodiment, S210 may include: determining a second azimuth depth map comprising the target class region and pixel position information of the target class region in the second azimuth depth map in the plurality of azimuth depth maps; the target class area is a pixel area presenting other observation surfaces in the second azimuth depth map; the other observation surfaces are observation surfaces corresponding to other azimuth texture maps except the second azimuth texture map in the plurality of azimuth texture maps; the second azimuth texture map is an azimuth texture map corresponding to the second azimuth depth map; determining depth information of the target class region in the second azimuth depth map according to pixel position information of the target class region in the second azimuth depth map; adjusting the observation parameters of a third azimuth depth map in the azimuth depth maps according to the depth information of the target class region to obtain an adjusted cube; re-observing the surface of the adjusted cube to obtain an adjusted azimuth depth map and an azimuth texture map; the area of the pixel area occupied by the target class area in the adjusted second azimuth depth map is smaller than that of the pixel area occupied by the target class area in the second azimuth depth map. The target class area may refer to an edge area of a target object, for example, the target object is an indoor target object, the target class area is a wall, the target object is an outdoor park c, and the target class area is an interface between park c and a non-park c.
The pixel position information of the target class region in the depth map can be determined according to the pixel position information of the target class region in the second azimuth texture map and the corresponding relation between the second azimuth texture map and the second azimuth depth map, and the pixel point corresponding to the target class region in the depth map is determined as the target class depth pixel point according to the pixel position information of the target class region in the second azimuth depth map; according to the depth value of the target class depth pixel point (namely the depth information of the target class area), the observation parameters of the third azimuth depth map are adjusted to adjust the distance between the observation surface corresponding to the third azimuth depth map and the preset view point corresponding to the third azimuth depth map, the adjustment of the cube surrounding the three-dimensional model is achieved, the adjusted cube is obtained, a plurality of new preset view points are determined according to the adjusted cube, a plurality of azimuth depth maps and a plurality of azimuth texture maps are determined according to the new preset view points, and the adjusted azimuth depth maps and the adjusted azimuth texture maps are used as the adjusted azimuth depth maps and the adjusted azimuth texture maps. Here, the adjusted azimuth depth map corresponds to the azimuth depth map one by one.
In the case where the second azimuth depth map may be an azimuth depth map observed at a front view point, the third azimuth depth map may be an azimuth depth map observed at least one of a left view point, a right view point, a top view point, and a bottom view point, the third azimuth depth map may be determined according to a position of the target class region in the second azimuth depth map. For example, if the position of the target class region in the second azimuth depth map is left, it is determined that the azimuth depth map observed by the left viewpoint is a third azimuth depth map, and if the position of the target class region in the second azimuth depth map is left and top, it is determined that the azimuth depth maps observed by the left viewpoint and the azimuth depth map observed by the top viewpoint are third azimuth depth maps. In the case where the second azimuth depth map is an azimuth depth map observed at the top view, the third azimuth depth map is an azimuth depth map observed at the rear view.
As described above, the distance between the observation surface corresponding to the depth map and the preset viewpoint corresponding to the depth map is inversely related to the observation parameter corresponding to the depth map, so the larger the area of the target class region is, the larger the area to be adjusted for eliminating (or reducing) the target class region is, the larger the observation parameter of the third-party depth map is, the smaller the area of the target class region is, and the smaller the area to be adjusted for eliminating (or reducing) the target class region is. When the observation parameters of the third azimuth depth map are adjusted according to the strategy, the area of the pixel region occupied by the target class region in the adjusted second azimuth depth map can be furthest enabled to be smaller than the area of the pixel region occupied by the target class region in the second azimuth depth map, and even the area of the target class region can be removed from the adjusted second azimuth depth map.
The observation surface corresponding to the third azimuth depth map surrounds the observation surface corresponding to the second azimuth depth map, so when the second azimuth depth map comprises the target area, the observation surface corresponding to the third azimuth depth map is included in the cube, and redundant information exists in the cube, so that the observation parameter corresponding to the third azimuth depth map needs to be adjusted to be increased, the volume of the cube is reduced, the adjusted cube is obtained, the adjusted cube does not comprise the observation surface corresponding to the third azimuth depth map or only comprises less part of the observation surface corresponding to the third azimuth depth map, and therefore the area (namely the target area) corresponding to the observation surface corresponding to the third azimuth depth map in the second azimuth depth map is reduced.
When the target class area is not included in all the azimuth depth maps, the observation parameters of the azimuth depth maps can not be adjusted.
It should be noted that the two adjustment modes of the depth map may be combined with each other, for example: the method comprises the steps of adjusting the observation parameters corresponding to the first azimuth depth map, adjusting the observation parameters of the third azimuth depth map, or adjusting the observation parameters of the third azimuth depth map, and adjusting the observation parameters corresponding to the first azimuth depth map.
Accordingly, in this embodiment, S140 may include: and generating an image according to the target viewpoint, the adjusted texture map, the adjusted depth map and the panoramic map, and obtaining a target image when the target object is watched from the target viewpoint.
And generating an image according to the target viewpoint, the adjusted texture map, the adjusted depth map and the panorama by an illusion engine (for example, UE 5) to obtain a target image when the target object is watched from the target viewpoint.
In some embodiments, S140 may include: combining the adjusted multiple azimuth depth maps into a combined depth map of multiple channels; the number of channels of the combined depth map is the same as the number of the adjusted azimuth depth maps; splicing a sixth azimuth texture map and a seventh azimuth texture map in the adjusted azimuth texture maps to obtain a spliced texture map; the sixth azimuth texture map is an adjusted azimuth texture map observed from the first preset viewpoint; the seventh azimuth texture map is an adjusted azimuth texture map observed from the second preset viewpoint; and generating a target image when the target object is watched from the target viewpoint according to the target viewpoint, the combined depth map, the spliced texture map and the panorama.
When the preset viewpoints include a positive viewpoint, a rear viewpoint, a left viewpoint, a right viewpoint, a rear viewpoint and a positive 45-degree viewpoint, the positive 45-degree viewpoint, the rear viewpoint and the viewpoints can be selected from the seven preset viewpoints as target preset viewpoints, and the adjusted azimuth depth maps observed by the target preset viewpoints are combined to obtain a combined depth map, wherein the number of channels of the combined depth map is the same as the number of channels of the target preset viewpoints (4 channels).
When the three-dimensional model of the target object is forward placed, the first preset viewpoint may be a forward viewpoint, and the second preset viewpoint may be a forward 45-degree viewpoint. The sixth azimuth texture map and the seventh azimuth texture map may be stitched in a horizontal direction: the sixth azimuthal texture map is stitched to the right of the seventh azimuthal texture map or the sixth azimuthal texture map is stitched to the left of the seventh azimuthal texture map.
Then, the image generation can be performed by the illusion engine (for example, UE 5) according to the target viewpoint, the merged depth map, the spliced texture map and the panorama, so as to obtain the target image when the three-dimensional model is viewed from the target viewpoint.
In this embodiment, the observation parameters of the depth map may be adjusted to obtain an adjusted cube, so that the adjusted depth map and the adjusted texture map obtained according to the adjusted cube include fewer target areas and shielding elements, and the adjusted depth map and the adjusted texture map are more accurate, so that the accuracy of the target image obtained according to the adjusted depth map and the adjusted texture map is higher.
In some embodiments, as shown in fig. 8, prior to step S210, the method may further include:
and S310, displaying the depth map and the texture map in a first image display area of the display interface.
The display interface further comprises a first image display area, and the first image display area comprises a first adjustment control corresponding to the depth map. As mentioned above, the depth map includes azimuth depth maps corresponding to each of the plurality of preset viewpoints, and the plurality of first adjustment controls are also provided, and one azimuth depth map corresponds to one first adjustment control.
When the target object is a room, as shown in fig. 9, a first image display area of the display interface is shown, and the first image display area 101 displays an azimuth depth map 10110 corresponding to a front view point, an azimuth texture map 10111 corresponding to a front view point, an azimuth depth map 10120 corresponding to a front 45-degree viewpoint, an azimuth texture map 10121 corresponding to a front 45-degree viewpoint, an azimuth depth map 10130 corresponding to a rear viewpoint, an azimuth texture map 10131 corresponding to a rear viewpoint, an azimuth depth map 10140 corresponding to a top viewpoint, and an azimuth texture map 10141 corresponding to a top viewpoint. The presentation interface 10 further includes respective first adjustment controls for each azimuthal depth map: the first adjustment control 1021 of the azimuth depth map corresponding to the forward viewpoint, the first adjustment control 1022 of the azimuth depth map corresponding to the backward viewpoint, the first adjustment control 1023 of the azimuth depth map corresponding to the left viewpoint, the first adjustment control 1024 of the azimuth depth map corresponding to the right viewpoint, the first adjustment control 1025 of the azimuth depth map corresponding to the bottom viewpoint, and the first adjustment control 1026 of the azimuth depth map corresponding to the top viewpoint.
Accordingly, S210 includes: responding to triggering operation for a first adjustment control, adjusting the observation parameters of the depth map into first observation parameters, and obtaining an adjusted cube; and re-observing the surface of the adjusted cube to obtain an adjusted azimuth depth map and an azimuth texture map.
The first observation parameter is trigger input. That is, the first observation parameter is set by the user according to the need. The first trigger operation may be an operation of the pointer on any first adjustment control.
For example, the first triggering operation may be input of a first observation parameter in the first adjustment control 1021 of the azimuth-depth map corresponding to the front viewpoint, and for another example, the first triggering operation may be a click operation of "+" or "-" in the first adjustment control 1022 of the azimuth-depth map corresponding to the rear viewpoint, so as to adjust an existing observation parameter in the first adjustment control 1022 to the first observation parameter.
After the first observation parameters are input, adjusting the observation parameters of the depth map according to the first observation parameters, and adjusting the cube surrounding the three-dimensional model to obtain an adjusted cube. Determining an adjusted depth map and a texture map according to the adjusted cube; the adjusted depth map comprises an adjusted azimuth depth map corresponding to each of the plurality of preset viewpoints, and the adjusted texture map comprises an azimuth texture map corresponding to each of the plurality of adjusted azimuth depth maps; one azimuth depth map corresponds to one adjusted azimuth depth map, and one adjusted azimuth depth map corresponds to one adjusted azimuth texture map.
As shown in fig. 9, whether the azimuth texture map 10111 corresponding to the forward view point in the display interface 10 is obviously blocked can be observed, and the lower part of the azimuth texture map 10111 corresponding to the forward view point is found to be blocked, and the blocking object is observed as a cabinet in combination with the azimuth texture map 10141 corresponding to the top view point, and meanwhile, the upper right part of the azimuth texture map 10111 corresponding to the forward view point is also partially blocked, which may be caused by uneven wall surface. The observed parameters of the azimuth texture map 10111 corresponding to the elevation points need to be adjusted to be large so that the cabinet part and the uneven wall surface model are excluded from the cube. At this time, as shown in fig. 10, the adjusted azimuth depth map corresponding to the forward-looking point is obtained, and compared with the azimuth texture map 10111 corresponding to the forward-looking point in fig. 10, the adjusted azimuth texture map 11111 corresponding to the forward-looking point in fig. 10 is no longer significantly occluded.
The azimuth depth map 11111 corresponding to the front view point in fig. 10 can be continuously observed, dark pixels are found at four edges of the image, and the wall surface of the three-dimensional model of the room is surrounded by the cube, so that the cube needs to be reduced, and the wall surface is excluded. The observation parameters of the azimuth depth map corresponding to each of the left viewpoint, the right viewpoint, the top viewpoint and the bottom viewpoint (the larger the dark pixel area is, the larger the increase amplitude is) can be appropriately increased, and the observation parameters are usually only required to be finely adjusted after decimal points. The adjusted depth map is shown in fig. 11, and the dark images of the four edges in the azimuth depth map 12111 corresponding to the front view point substantially disappear.
The azimuth depth map 12140 corresponding to the top view point in fig. 11 may be continuously observed, and the upper boundary dark area of the azimuth depth map 12140 corresponding to the top view point is found to be more, so that the wall surface of the bright room model is determined to be surrounded by the cube, and the cube needs to be reduced, so that the model wall surface is excluded. The observation parameters of the azimuth depth map 12130 corresponding to the rear view point may be appropriately increased (the larger the dark pixel area, the larger the magnitude of the increase). As shown in fig. 12, the depth map after adjustment reduces dark pixels at the upper edge in the azimuth depth map 13141 corresponding to the top view point.
After S210, the method may further include:
and S320, displaying the adjusted depth map and the adjusted texture map in the first image display area.
After the adjusted azimuth depth map is obtained, the first image display area can also directly output the adjusted azimuth depth map.
As shown in fig. 12, after the depth map shown in fig. 9 is adjusted as described above, the adjusted depth map is displayed in the first image display area 101: a forward viewpoint corresponding azimuth depth map 13110, a forward 45 degree viewpoint corresponding azimuth depth map 13120, a backward viewpoint corresponding azimuth depth map 13130, and a top viewpoint corresponding azimuth depth map 13140.
After the depth map is adjusted, the texture map can be adjusted according to the adjusted depth map, so that an adjusted texture map is obtained, and the adjusted texture map is displayed in the first image display area. As shown in fig. 13, the adjusted texture map may be displayed in the first image display area 101: a forward viewpoint corresponding azimuth texture map 13111, a forward 45 degree viewpoint corresponding azimuth texture map 13121, a backward viewpoint corresponding azimuth texture map 13131, and a top viewpoint corresponding azimuth texture map 13141.
It should be noted that after the texture map corresponding to the depth map is adjusted, the adjusted texture map and the adjusted depth map may be output at the first image display interface before S140.
In this embodiment, the user may manually modify the observation parameters of the depth map and the observation parameters of the texture map based on the requirements, so as to obtain an adjusted depth map and an adjusted texture map including fewer target areas and shielding elements, where the adjusted depth map and the adjusted texture map are more compatible with the requirements of the user, and further the target image obtained according to the adjusted depth map and the adjusted texture map has a higher degree of compliance with the requirements of the user.
Secondly, in the whole manufacturing process, only the uploading of training samples and the fine adjustment of the images (depth image and texture image) need to be manually participated, and the art asset can be efficiently produced under the condition that input data is ready. Meanwhile, the mode of efficiently and semi-automatically generating the art asset can greatly reduce the modeling workload and time consumption of the art staff and save the cost.
In some embodiments, as shown in fig. 13, after S120, the method may further include:
s410, adjusting the observation parameters of the target cube map in the plurality of cube maps to obtain the adjusted target cube map.
The target observation result also comprises observation parameters corresponding to each cube map; the observation parameters of the cube map are used to indicate the focal length of the viewpoint to which the cube map corresponds. The focal length of the viewpoint of the cube map corresponding to the observed parameter of the cube map may be a positive correlation: the larger the observation parameters of the cube map are, the larger the focal length of the view point corresponding to the cube map is, the smaller the observation parameters of the cube map are, and the smaller the focal length of the view point corresponding to the cube map is.
The target cube map is a cube map with definition smaller than a preset definition threshold. The preset sharpness threshold may be a value set based on requirements, which is not limited in this application.
When the target cube map is a cube map with a sharpness less than the preset sharpness threshold, S410 may include: according to the definition of the target cube map, adjusting the observation parameters of the target cube map to obtain an adjusted target cube map; the sharpness of the target cube map is lower than the sharpness of the adjusted target cube map.
When the definition of the cube map is lower, the focal length of the view point corresponding to the cube map needs to be adjusted down, at this time, the observation parameters of the cube map can be adjusted down, and when the definition is improved, the focal length of the view point corresponding to the cube map needs to be improved, and the observation parameters of the cube map can be adjusted up.
After the target cube map is determined, the observation parameters of the target cube map can be adjusted according to the definition difference value between the definition of the target cube map and the preset definition threshold value, so that the adjusted target cube map is obtained. The larger the sharpness difference, the larger the magnitude of the undershoot of the observed parameter of the target cube map.
Accordingly, S130 may include: and fusing the adjusted target cube map and other cube maps except the target cube map in the plurality of cube maps to obtain a panoramic image presenting the three-dimensional model.
In this embodiment, a panoramic view of the three-dimensional model may be generated by fusing the cube2sphere according to the adjusted target cube map and other cube maps in the plurality of cube maps except for the target cube map.
In this embodiment, the cube map may be adjusted, so that the sharpness of the adjusted cube map is higher, thereby improving the sharpness of the obtained panoramic map, and further, the accuracy of the target image obtained according to the panoramic map is higher.
In some embodiments, as shown in fig. 14, prior to S410, the method may further include:
s510, displaying a plurality of cube maps in a second image display area of the display interface.
The display interface also comprises a second image display area, and the second image display area comprises a second adjustment control corresponding to each cube map; as mentioned above, the number of the cube maps is plural, and the number of the second adjustment controls is plural, and one cube map corresponds to one second adjustment control.
When the target object is a room, as shown in fig. 15, the display interface 10 includes a second image display area 16, and the second image display area 16 displays an orthocube map 1611 observed from the center of the cube to the front viewing surface, a rear cube map 1612 observed from the center of the cube to the rear viewing surface, a left cube map 1613 observed from the center of the cube to the left viewing surface, a right cube map 1614 observed from the center of the cube to the right viewing surface, a bottom cube map 1615 observed from the center of the cube to the bottom viewing surface, and a top cube map 1616 observed from the center of the cube to the top viewing surface, as shown in fig. 15. The second image display area 16 may also display a second adjustment control 1621 corresponding to the cube map 1611, a second adjustment control 1622 corresponding to the rear cube map 1612, a second adjustment control 1623 corresponding to the left cube map 1613, a second adjustment control 1624 corresponding to the right cube map 1614, a second adjustment control 1625 corresponding to the bottom cube map 1615, and a second adjustment control 1626 corresponding to the top cube map 1616.
It should be noted that, in some embodiments, the first image display area and the second image display area may be two different areas in the display interface; or, the display interface can display the first image display area firstly, and then display the second image display area after finishing the adjustment of the observation parameters of the depth map and the texture map so as to finish the adjustment of the observation parameters of the cube map; or, the display interface may display the second image display area first to complete adjustment of the observation parameters of the cube map, and then display the first image display area to complete adjustment of the observation parameters of the depth map and the texture map.
Accordingly, S410 may include: and adjusting the observation parameters of the target cube map to second observation parameters in response to the triggering operation of a second adjustment control corresponding to the target cube map, so as to obtain the adjusted target cube map.
The second observation parameter is trigger input. That is, the second observation parameter is set by the user according to the need. Wherein the second trigger operation may be an operation of the pointer on any second adjustment control.
For example, the second trigger operation may be entering a second observed parameter within the second adjustment control 1621 of the cube map 1611, and for example, the second trigger operation may be a click operation on "+" or "-" within the second adjustment control 1622 of the rear cube map to adjust an observed parameter existing within the second adjustment control 1622 to the second observed parameter.
After S410, the method may further include:
and S520, displaying the adjusted target cube map in a second image display area of the display interface.
After the adjusted azimuth depth map is obtained, the adjusted target cube map can be directly displayed in the second image display area. Meanwhile, other cube maps than the target cube map among the plurality of cube maps may also be displayed.
As shown in fig. 16, after the cube map shown in fig. 15 is adjusted as described above, the adjusted cube map is displayed in the second image display area 16: an orthocube map 1711 viewed from the center of the cube to the positive viewing surface, a back cube map 1712 viewed from the center of the cube to the back viewing surface, a left cube map 1713 viewed from the center of the cube to the left viewing surface, a right cube map 1714 viewed from the center of the cube to the right viewing surface, a bottom cube map 1715 viewed from the center of the cube to the bottom viewing surface, and a top cube map 1716 viewed from the center of the cube to the top viewing surface.
When the target object is a room, the adjusted depth map, the adjusted texture map, and the adjusted cube map may be determined according to the foregoing embodiments of fig. 7 to 13, and then the target image obtained according to the adjusted depth map, the adjusted texture map, and the adjusted cube map is shown in fig. 17.
It should be noted that after the texture map corresponding to the depth map is adjusted, the adjusted cube map may be output at the second image display interface before S130.
In this embodiment, as shown in fig. 18, the process of acquiring the target image is to acquire training samples (a plurality of sample viewpoints and a sample shooting image obtained by shooting the target object at each sample viewpoint) through data acquisition, create tasks through a task interface to import a picture set (sample shooting image), and a background algorithm performs three-dimensional model construction according to the imported picture set to obtain a three-dimensional model, and meanwhile, can also retain data (in the form of a training log) of the training process. After the training is performed to obtain the three-dimensional model, task details can be checked through a task creation interface so as to connect the training process and the three-dimensional model obtained by training.
The model prediction is directly performed according to the trained three-dimensional model to obtain a prediction result (namely, the target observation result in the embodiment), the target observation result is derived, namely, the target observation result is displayed on the display interface, the observation parameters of each image in the display interface can be manually adjusted so as to update the model parameters according to the adjusted observation parameters, an adjusted cube is obtained, an adjusted image (comprising an adjusted depth map, an adjusted texture map and an adjusted cube map) is obtained according to the adjusted cube, and finally, the target image is obtained according to the adjusted image input into the illusion engine.
In this embodiment, the user may manually modify the observation parameters of the depth map and the observation parameters of the texture map based on the requirements, so as to obtain an adjusted depth map and an adjusted texture map including fewer target areas and shielding elements, where the adjusted depth map and the adjusted texture map are more compatible with the requirements of the user, and further the target image obtained according to the adjusted depth map and the adjusted texture map has a higher degree of compliance with the requirements of the user.
Referring to fig. 19, fig. 19 shows a block diagram of an image generating apparatus according to an embodiment of the present application, where the apparatus 1900 includes:
an obtaining module 1910, configured to perform three-dimensional reconstruction on a sample photographed image obtained by photographing a target object according to a plurality of sample viewpoints and each sample viewpoint, to obtain a three-dimensional model of the target object;
a determining module 1920, configured to observe the three-dimensional model, and obtain a target observation result corresponding to the target object; the target observation result comprises a plurality of cube maps, texture maps and depth maps corresponding to the texture maps;
the fusion module 1930 is used for fusing the plurality of cube maps to obtain a panoramic image presenting the three-dimensional model;
the generating module 1940 is configured to generate an image according to the target viewpoint, the texture map, the depth map, and the panorama, so as to obtain a target image when the target object is viewed from the target viewpoint.
Optionally, the texture map comprises a plurality of azimuthal texture maps observed from a plurality of preset viewpoints; the depth map comprises azimuth depth maps respectively corresponding to the azimuth texture maps; a determination module 1920 further configured to determine a cube surrounding the three-dimensional model; determining a plurality of preset viewpoints according to the positions of the cubes; observing the surface of the cube from a plurality of preset viewpoints respectively to obtain a plurality of azimuth texture maps and azimuth depth maps corresponding to each azimuth texture map; and observing the space surrounded by the cube along the direction perpendicular to each surface of the cube by taking the center of the cube as an observation position to obtain a plurality of cube maps.
Optionally, the target observation result further includes an observation parameter corresponding to the depth map; the observation parameters of the depth map are used for indicating the distance between the observation surface corresponding to the depth map and the preset viewpoint corresponding to the depth map; the observation surface is a surface on a cube surrounding the three-dimensional model; the device also comprises an adjusting module, a display module and a display module, wherein the adjusting module is used for adjusting the observation parameters corresponding to the depth map to obtain an adjusted cube; determining an adjusted depth map and an adjusted texture map according to the adjusted cube; correspondingly, the generating module 1940 is further configured to generate an image according to the target viewpoint, the adjusted texture map, the adjusted depth map, and the panorama, so as to obtain a target image when the target object is viewed from the target viewpoint.
Optionally, the texture map comprises azimuth texture maps respectively observed from a plurality of preset viewpoints; the depth map comprises azimuth depth maps respectively corresponding to the azimuth texture maps; the adjusted texture map comprises an adjusted azimuth texture map corresponding to each azimuth texture map; the adjusted depth map comprises an adjusted azimuth depth map corresponding to each azimuth depth map; the adjusting module is also used for determining a first azimuth texture map in which the target object is blocked by the blocking element and pixel position information of the blocking element in the first azimuth texture map in the plurality of azimuth texture maps; determining depth information of the shielding element in a first azimuth depth map corresponding to the first azimuth texture map according to pixel position information of the shielding element in the first azimuth texture map; adjusting the observation parameters of the first azimuth depth map according to the depth information of the shielding element to obtain an adjusted cube; re-observing the surface of the adjusted cube to obtain an adjusted azimuth depth map and an azimuth texture map; the area of the pixel area occupied by the shielding element in the adjusted first azimuth depth map is smaller than that of the pixel area occupied in the first azimuth depth map.
Optionally, the texture map comprises azimuth texture maps respectively observed from a plurality of preset viewpoints; the depth map comprises azimuth depth maps respectively corresponding to the azimuth texture maps; the adjusted texture map comprises an adjusted azimuth texture map corresponding to each azimuth texture map; the adjusted depth map comprises an adjusted azimuth depth map corresponding to each azimuth depth map; the adjusting module is further used for determining a second azimuth depth map comprising the target class area and pixel position information of the target class area in the second azimuth depth map from the plurality of azimuth depth maps; the target class area is a pixel area presenting other observation surfaces in the second azimuth depth map; the other observation surfaces are observation surfaces corresponding to other azimuth texture maps except the second azimuth texture map in the plurality of azimuth texture maps; the second azimuth texture map is an azimuth texture map corresponding to the second azimuth depth map; determining depth information of the target class region in the second azimuth depth map according to pixel position information of the target class region in the second azimuth depth map; adjusting the observation parameters of a third azimuth depth map in the azimuth depth maps according to the depth information of the target class region to obtain an adjusted cube; re-observing the surface of the adjusted cube to obtain an adjusted azimuth depth map and an azimuth texture map; the area of the pixel area occupied by the target class area in the adjusted second azimuth depth map is smaller than that of the pixel area occupied by the target class area in the second azimuth depth map.
Optionally, the adjusting module is further configured to adjust an observation parameter of the depth map to a first observation parameter in response to a triggering operation for the first adjusting control, so as to obtain an adjusted cube; the first observation parameter is input in a triggering way; re-observing the surface of the adjusted cube to obtain an adjusted azimuth depth map and an azimuth texture map; and displaying the adjusted depth map and the adjusted texture map in the first image display area.
Optionally, the target observation result further comprises observation parameters corresponding to each cube map; the observation parameters of the cube map are used for indicating the focal length of the view point corresponding to the cube map; the adjusting module is also used for adjusting the observation parameters of the target cube map in the plurality of cube maps to obtain an adjusted target cube map; correspondingly, the fusion module 1930 is further configured to fuse the adjusted target cube map and other cube maps except for the target cube map in the plurality of cube maps, so as to obtain a panorama that presents the three-dimensional model.
Optionally, the target cube map is a cube map having a sharpness less than a preset sharpness threshold; the adjusting module is also used for adjusting the observation parameters of the target cube map according to the definition of the target cube map to obtain an adjusted target cube map; the sharpness of the target cube map is lower than the sharpness of the adjusted target cube map.
Optionally, the adjustment module is further configured to display a plurality of cube maps in a second image display area of the display interface; the second display interface also comprises a second image display area, and the second image display area comprises a second adjustment control corresponding to each cube map; responding to the triggering operation of a second adjustment control corresponding to the target cube map, and adjusting the observation parameters of the target cube map to second observation parameters to obtain an adjusted target cube map; the second observation parameter is input in a triggering way; and displaying the adjusted target cube map in a second image display area of the display interface.
Optionally, the texture map comprises a plurality of azimuthal texture maps observed from a plurality of preset viewpoints; the depth map comprises azimuth depth maps respectively corresponding to the azimuth texture maps; the generating module 1940 is further configured to combine the multiple azimuth depth maps into a combined depth map of multiple channels; the number of channels of the combined depth map is the same as the number of azimuth depth maps; splicing a fourth azimuth texture map and a fifth azimuth texture map in the azimuth texture maps to obtain a spliced texture map; the fourth azimuth texture map is an azimuth texture map observed from the first preset viewpoint; the fifth azimuth texture map is an azimuth texture map observed from a second preset viewpoint; and generating a target image when the target object is watched from the target viewpoint according to the target viewpoint, the combined depth map, the spliced texture map and the panorama.
Optionally, the apparatus further comprises a sample acquisition module for displaying a task creation interface; the task creation interface includes a selection control; and responding to the selection operation of the selection control, acquiring a plurality of selected sample viewpoints and sample shooting images obtained by shooting the target object at each sample viewpoint.
It should be noted that, in the present application, the device embodiment and the foregoing method embodiment correspond to each other, and specific principles in the device embodiment may refer to the content in the foregoing method embodiment, which is not described herein again.
Fig. 20 shows a block diagram of a configuration of an electronic device for executing an image generating method according to an embodiment of the present application. The electronic device may be the terminal 20 or the server 10 in fig. 1, and it should be noted that, the computer system 1200 of the electronic device shown in fig. 20 is only an example, and should not impose any limitation on the functions and the application scope of the embodiments of the present application.
As shown in fig. 20, the computer system 1200 includes a central processing unit (Central Processing Unit, CPU) 1201 which can perform various appropriate actions and processes, such as performing the methods in the above-described embodiments, according to a program stored in a Read-Only Memory (ROM) 1202 or a program loaded from a storage section 1208 into a random access Memory (Random Access Memory, RAM) 1203. In the RAM 1203, various programs and data required for the system operation are also stored. The CPU1201, ROM1202, and RAM 1203 are connected to each other through a bus 1204. An Input/Output (I/O) interface 1205 is also connected to bus 1204.
The following components are connected to the I/O interface 1205: an input section 1206 including a keyboard, a mouse, and the like; an output portion 1207 including a Cathode Ray Tube (CRT), a liquid crystal display (Liquid Crystal Display, LCD), and a speaker, etc.; a storage section 1208 including a hard disk or the like; and a communication section 1209 including a network interface card such as a LAN (Local Area Network ) card, a modem, or the like. The communication section 1209 performs communication processing via a network such as the internet. The drive 1210 is also connected to the I/O interface 1205 as needed. A removable medium 1211 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed on the drive 1210 as needed, so that a computer program read out therefrom is installed into the storage section 1208 as needed.
In particular, according to embodiments of the present application, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present application include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 1209, and/or installed from the removable media 1211. When executed by a Central Processing Unit (CPU) 1201, performs the various functions defined in the system of the present application.
It should be noted that, the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-Only Memory (ROM), an erasable programmable read-Only Memory (Erasable Programmable Read Only Memory, EPROM), flash Memory, an optical fiber, a portable compact disc read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. Where each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present application may be implemented by means of software, or may be implemented by means of hardware, and the described units may also be provided in a processor. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
As another aspect, the present application also provides a computer-readable storage medium that may be included in the electronic device described in the above embodiments; or may exist alone without being incorporated into the electronic device. The computer readable storage medium carries computer readable instructions which, when executed by a processor, implement the method of any of the above embodiments.
According to an aspect of embodiments of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the electronic device to perform the method of any of the embodiments described above.
It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit, in accordance with embodiments of the present application. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.
From the above description of embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented in software, or may be implemented in software in combination with the necessary hardware. Thus, the technical solution according to the embodiments of the present application may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (may be a CD-ROM, a usb disk, a mobile hard disk, etc.) or on a network, and includes several instructions to cause an electronic device (may be a personal computer, a server, a touch terminal, or a network device, etc.) to perform the method according to the embodiments of the present application.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting thereof; although the present application has been described in detail with reference to the foregoing embodiments, one of ordinary skill in the art will appreciate that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not drive the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (12)
1. An image generation method, the method comprising:
carrying out three-dimensional reconstruction on a sample shooting image obtained by shooting a target object according to a plurality of sample viewpoints and each sample viewpoint to obtain a three-dimensional model of the target object;
observing the three-dimensional model to obtain a target observation result corresponding to the target object; the target observation result comprises a plurality of cube maps, texture maps, depth maps corresponding to the texture maps and observation parameters corresponding to the depth maps; the observation parameters of the depth map are used for indicating the distance between the observation surface corresponding to the depth map and the preset viewpoint corresponding to the depth map; the observation surface corresponding to the depth map is a surface on a cube surrounding the three-dimensional model;
Fusing a plurality of the cube maps to obtain a panoramic image presenting the three-dimensional model;
adjusting observation parameters corresponding to the depth map to obtain an adjusted cube, and determining an adjusted depth map and an adjusted texture map according to the adjusted cube;
generating an image according to a target viewpoint, the adjusted texture map, the adjusted depth map and the panoramic map, and obtaining a target image when the target object is watched from the target viewpoint;
the texture map comprises a plurality of azimuth texture maps respectively observed from a plurality of preset viewpoints; the depth map comprises azimuth depth maps respectively corresponding to a plurality of azimuth texture maps; the adjusted texture map comprises an adjusted azimuth texture map corresponding to each azimuth texture map; the adjusted depth map comprises an adjusted azimuth depth map corresponding to each azimuth depth map; the adjusting the observation parameters corresponding to the depth map to obtain an adjusted cube, and determining an adjusted depth map and an adjusted texture map according to the adjusted cube, including:
determining a first azimuth texture map in which a target object is blocked by a blocking element and pixel position information of the blocking element in the first azimuth texture map in a plurality of azimuth texture maps; determining depth information of the shielding element in a first azimuth depth map corresponding to the first azimuth texture map according to pixel position information of the shielding element in the first azimuth texture map; adjusting the observation parameters of the first azimuth depth map according to the depth information of the shielding elements to obtain an adjusted cube;
Re-observing the surface of the adjusted cube to obtain an adjusted azimuth depth map and an azimuth texture map; the area of the pixel area occupied by the shielding element in the adjusted first azimuth depth map is smaller than that of the pixel area occupied in the first azimuth depth map.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the observing the three-dimensional model to obtain a target observation result corresponding to the target object comprises the following steps:
determining a cube surrounding the three-dimensional model;
determining a plurality of preset viewpoints according to the positions of the cubes;
observing the surface of the cube from the plurality of preset viewpoints respectively to obtain a plurality of azimuth texture maps and azimuth depth maps corresponding to each azimuth texture map;
and observing the space surrounded by the cube along the direction perpendicular to each surface on the cube by taking the center of the cube as an observation position to obtain a plurality of cube maps.
3. The method of claim 1, wherein the step of determining the position of the substrate comprises,
the observation parameters corresponding to the depth map are adjusted to obtain an adjusted cube, and the adjusted depth map and the adjusted texture map are determined according to the adjusted cube and can be replaced by:
Determining a second azimuth depth map comprising a target class region and pixel position information of the target class region in the second azimuth depth map in a plurality of azimuth depth maps; the target class area refers to a pixel area presenting other observation surfaces in the second azimuth depth map; the other observation surfaces are observation surfaces corresponding to other azimuth texture maps except the second azimuth texture map in the azimuth texture maps; the second azimuth texture map is an azimuth texture map corresponding to the second azimuth depth map;
determining depth information of the target class region in the second azimuth depth map according to pixel position information of the target class region in the second azimuth depth map;
adjusting the observation parameters of a third azimuth depth map in the azimuth depth maps according to the depth information of the target class region to obtain an adjusted cube;
re-observing the surface of the adjusted cube to obtain an adjusted azimuth depth map and an adjusted azimuth texture map; and the area of the pixel area occupied by the target class area in the adjusted second azimuth depth map is smaller than the area of the pixel area occupied by the target class area in the second azimuth depth map.
4. The method of claim 1, wherein before adjusting the observation parameters corresponding to the depth map to obtain the adjusted cube, the method further comprises:
displaying the depth map and the texture map in a first image display area of a display interface; the display interface further comprises a first image display area, wherein the first image display area comprises a first adjustment control corresponding to the depth map;
the adjusting the observation parameters corresponding to the depth map to obtain an adjusted cube further comprises:
responding to the triggering operation of the first adjusting control, adjusting the observation parameters of the depth map to be first observation parameters, and obtaining an adjusted cube; the first observation parameter is input in a triggering way;
the method further comprises the steps of:
and displaying the adjusted depth map and the adjusted texture map in the first image display area.
5. The method of claim 1 or 2, wherein the target observations further comprise observation parameters corresponding to each of the cube maps; the observation parameters of the cube map are used for indicating the focal length of the view point corresponding to the cube map;
After the three-dimensional model is observed to obtain the target observation result corresponding to the target object, the method further comprises the following steps:
adjusting the observation parameters of the target cube maps in the plurality of cube maps to obtain adjusted target cube maps;
the fusing of the plurality of cube maps to obtain a panoramic image presenting the three-dimensional model comprises the following steps:
and fusing the adjusted target cube map and other cube maps except the target cube map in the plurality of cube maps to obtain a panoramic image presenting the three-dimensional model.
6. The method of claim 5, wherein the target cube map is a cube map having a sharpness less than a preset sharpness threshold;
the adjusting the observation parameters of the target cube map in the plurality of cube maps to obtain an adjusted target cube map comprises:
according to the definition of the target cube map, adjusting the observation parameters of the target cube map to obtain an adjusted target cube map; the sharpness of the target cube map is lower than the sharpness of the adjusted target cube map.
7. The method of claim 5, wherein adjusting the observed parameters of the target cube map of the plurality of cube maps, prior to obtaining the adjusted target cube map, further comprises:
displaying the plurality of cube maps in a second image display area of the display interface; the second display interface further comprises a second image display area, and the second image display area comprises a second adjustment control corresponding to each cube map;
the adjusting the observation parameters of the target cube map in the plurality of cube maps to obtain an adjusted target cube map comprises:
responding to triggering operation of a second adjustment control corresponding to the target cube map, and adjusting the observation parameters of the target cube map to second observation parameters to obtain an adjusted target cube map; the second observation parameter is input in a triggering way;
the method further comprises the steps of:
and displaying the adjusted target cube map in a second image display area of the display interface.
8. The method of claim 1, wherein the adjusted texture comprises a plurality of adjusted azimuth texture maps observed from a plurality of preset viewpoints; the adjusted depth map comprises a plurality of adjusted azimuth depth maps respectively corresponding to the adjusted azimuth texture maps;
Generating an image according to the target viewpoint, the adjusted texture map, the adjusted depth map and the panorama map to obtain a target image when the target object is watched from the target viewpoint, including:
combining the plurality of adjusted azimuth depth maps into a combined depth map of multiple channels; the number of channels of the combined depth map is the same as the number of the adjusted azimuth depth maps;
splicing a sixth azimuth texture map and a seventh azimuth texture map in the plurality of adjusted azimuth texture maps to obtain spliced texture maps; the sixth azimuth texture map is an adjusted azimuth texture map observed from a first preset viewpoint; the seventh azimuth texture map is an adjusted azimuth texture map observed from a second preset viewpoint;
and generating a target image when the target object is watched from the target viewpoint according to the target viewpoint, the combined depth map, the spliced texture map and the panorama.
9. The method according to claim 1, wherein the three-dimensional reconstruction is performed according to a plurality of sample viewpoints and sample photographed images obtained by photographing the target object at each of the sample viewpoints, and before obtaining the three-dimensional model of the target object, the method further comprises:
Displaying a task creation interface; the task creation interface comprises a selection control;
and responding to the selection operation of the selection control, acquiring a plurality of selected sample viewpoints and sample shooting images obtained by shooting the target object at each sample viewpoint.
10. An image generation apparatus, the apparatus comprising:
the acquisition module is used for carrying out three-dimensional reconstruction on a sample shooting image obtained by shooting a target object according to a plurality of sample viewpoints and each sample viewpoint to obtain a three-dimensional model of the target object;
the determining module is used for observing the three-dimensional model to obtain a target observation result corresponding to the target object; the target observation result comprises a plurality of cube maps, texture maps, depth maps corresponding to the texture maps and observation parameters corresponding to the depth maps; the observation parameters of the depth map are used for indicating the distance between the observation surface corresponding to the depth map and the preset viewpoint corresponding to the depth map; the observation surface corresponding to the depth map is a surface on a cube surrounding the three-dimensional model;
the fusion module is used for fusing the plurality of cube patches to obtain a panoramic image showing the three-dimensional model; adjusting observation parameters corresponding to the depth map to obtain an adjusted cube, and determining an adjusted depth map and an adjusted texture map according to the adjusted cube;
The generation module is used for generating images according to the target viewpoint, the adjusted texture map, the adjusted depth map and the panoramic map to obtain a target image when the target object is watched from the target viewpoint;
the texture map comprises a plurality of azimuth texture maps respectively observed from a plurality of preset viewpoints; the depth map comprises azimuth depth maps respectively corresponding to a plurality of azimuth texture maps; the adjusted texture map comprises an adjusted azimuth texture map corresponding to each azimuth texture map; the adjusted depth map comprises an adjusted azimuth depth map corresponding to each azimuth depth map; the fusion module is further used for determining a first azimuth texture map, which is blocked by the blocking element, of the target object and pixel position information of the blocking element in the first azimuth texture map in the multiple azimuth texture maps; determining depth information of the shielding element in a first azimuth depth map corresponding to the first azimuth texture map according to pixel position information of the shielding element in the first azimuth texture map; adjusting the observation parameters of the first azimuth depth map according to the depth information of the shielding elements to obtain an adjusted cube; re-observing the surface of the adjusted cube to obtain an adjusted azimuth depth map and an azimuth texture map; the area of the pixel area occupied by the shielding element in the adjusted first azimuth depth map is smaller than that of the pixel area occupied in the first azimuth depth map.
11. An electronic device, comprising:
a processor;
a memory having stored thereon computer readable instructions which, when executed by the processor, implement the method of any of claims 1-9.
12. A computer readable storage medium having computer readable instructions stored thereon, which when executed by a processor, implement the method of any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311472485.XA CN117197319B (en) | 2023-11-07 | 2023-11-07 | Image generation method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311472485.XA CN117197319B (en) | 2023-11-07 | 2023-11-07 | Image generation method, device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117197319A CN117197319A (en) | 2023-12-08 |
| CN117197319B true CN117197319B (en) | 2024-03-22 |
Family
ID=88985494
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311472485.XA Active CN117197319B (en) | 2023-11-07 | 2023-11-07 | Image generation method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117197319B (en) |
Citations (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101686334A (en) * | 2008-09-28 | 2010-03-31 | 新奥特（北京）视频技术有限公司 | Method and device for acquiring three-dimensional image scene |
| CN104463959A (en) * | 2014-11-25 | 2015-03-25 | 无锡梵天信息技术股份有限公司 | Method for generating cubic environment maps |
| JP2018049591A (en) * | 2016-09-14 | 2018-03-29 | キヤノン株式会社 | Image processing apparatus, image generation method, and program |
| CN111462311A (en) * | 2020-03-31 | 2020-07-28 | 北京小米松果电子有限公司 | Panorama generation method and device and storage medium |
| CN111669564A (en) * | 2019-03-07 | 2020-09-15 | 阿里巴巴集团控股有限公司 | Image reconstruction method, system, device and computer readable storage medium |
| CN113096008A (en) * | 2021-03-09 | 2021-07-09 | 惠州Tcl移动通信有限公司 | Panoramic picture display method, display device and storage medium |
| CN113129417A (en) * | 2019-12-27 | 2021-07-16 | 华为技术有限公司 | Image rendering method in panoramic application and terminal equipment |
| CN113221847A (en) * | 2021-06-07 | 2021-08-06 | 广州虎牙科技有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
| JP2021157237A (en) * | 2020-03-25 | 2021-10-07 | Kddi株式会社 | Free viewpoint video generation method, device and program |
| CN113891060A (en) * | 2020-07-03 | 2022-01-04 | 阿里巴巴集团控股有限公司 | Free viewpoint video reconstruction method, playing processing method, device and storage medium |
| WO2022001865A1 (en) * | 2020-07-03 | 2022-01-06 | 阿里巴巴集团控股有限公司 | Depth map and video processing and reconstruction methods and apparatuses, device, and storage medium |
| CN114241127A (en) * | 2021-12-01 | 2022-03-25 | 维沃移动通信有限公司 | Panoramic image generation method and device, electronic equipment and medium |
| CN114549731A (en) * | 2022-04-22 | 2022-05-27 | 清华大学 | Method and device for generating visual angle image, electronic equipment and storage medium |
| CN116012432A (en) * | 2023-01-18 | 2023-04-25 | 广州南天电脑系统有限公司 | Stereoscopic panoramic image generation method and device and computer equipment |
| CN116310046A (en) * | 2023-05-16 | 2023-06-23 | 腾讯科技（深圳）有限公司 | Image processing method, device, computer and storage medium |
| CN116310036A (en) * | 2023-03-31 | 2023-06-23 | 北京字跳网络技术有限公司 | Scene rendering method, device, equipment, computer readable storage medium and product |
| CN116501209A (en) * | 2023-05-09 | 2023-07-28 | 网易（杭州）网络有限公司 | Editing view angle adjusting method and device, electronic equipment and readable storage medium |
| CN116740261A (en) * | 2022-03-02 | 2023-09-12 | 腾讯科技（深圳）有限公司 | Image reconstruction method and device and training method and device of image reconstruction model |
| CN116958378A (en) * | 2023-02-21 | 2023-10-27 | 腾讯科技（深圳）有限公司 | Face map reconstruction method, device, computer readable medium and electronic equipment |
| CN116977392A (en) * | 2022-12-29 | 2023-10-31 | 腾讯科技（深圳）有限公司 | Image generation method, device, electronic equipment and storage medium |
| CN116977531A (en) * | 2023-07-28 | 2023-10-31 | 腾讯科技（深圳）有限公司 | Three-dimensional texture image generation method, three-dimensional texture image generation device, computer equipment and storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102971770B (en) * | 2011-03-31 | 2016-02-10 | 松下电器产业株式会社 | Carry out the all-round image displaying device, the image drawing method that enclose the description of stereo-picture |
| US9055277B2 (en) * | 2011-03-31 | 2015-06-09 | Panasonic Intellectual Property Management Co., Ltd. | Image rendering device, image rendering method, and image rendering program for rendering stereoscopic images |
| US10165259B2 (en) * | 2017-02-15 | 2018-12-25 | Adobe Systems Incorporated | Generating novel views of a three-dimensional object based on a single two-dimensional image |
| CN110136082B (en) * | 2019-05-10 | 2021-06-01 | 腾讯科技（深圳）有限公司 | Occlusion rejection method and device and computer equipment |
-
2023
- 2023-11-07 CN CN202311472485.XA patent/CN117197319B/en active Active
Patent Citations (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101686334A (en) * | 2008-09-28 | 2010-03-31 | 新奥特（北京）视频技术有限公司 | Method and device for acquiring three-dimensional image scene |
| CN104463959A (en) * | 2014-11-25 | 2015-03-25 | 无锡梵天信息技术股份有限公司 | Method for generating cubic environment maps |
| JP2018049591A (en) * | 2016-09-14 | 2018-03-29 | キヤノン株式会社 | Image processing apparatus, image generation method, and program |
| CN111669564A (en) * | 2019-03-07 | 2020-09-15 | 阿里巴巴集团控股有限公司 | Image reconstruction method, system, device and computer readable storage medium |
| CN113129417A (en) * | 2019-12-27 | 2021-07-16 | 华为技术有限公司 | Image rendering method in panoramic application and terminal equipment |
| JP2021157237A (en) * | 2020-03-25 | 2021-10-07 | Kddi株式会社 | Free viewpoint video generation method, device and program |
| CN111462311A (en) * | 2020-03-31 | 2020-07-28 | 北京小米松果电子有限公司 | Panorama generation method and device and storage medium |
| WO2022002181A1 (en) * | 2020-07-03 | 2022-01-06 | 阿里巴巴集团控股有限公司 | Free viewpoint video reconstruction method and playing processing method, and device and storage medium |
| CN113963094A (en) * | 2020-07-03 | 2022-01-21 | 阿里巴巴集团控股有限公司 | Depth map and video processing and reconstruction method, device, equipment and storage medium |
| CN113891060A (en) * | 2020-07-03 | 2022-01-04 | 阿里巴巴集团控股有限公司 | Free viewpoint video reconstruction method, playing processing method, device and storage medium |
| WO2022001865A1 (en) * | 2020-07-03 | 2022-01-06 | 阿里巴巴集团控股有限公司 | Depth map and video processing and reconstruction methods and apparatuses, device, and storage medium |
| CN113096008A (en) * | 2021-03-09 | 2021-07-09 | 惠州Tcl移动通信有限公司 | Panoramic picture display method, display device and storage medium |
| CN113221847A (en) * | 2021-06-07 | 2021-08-06 | 广州虎牙科技有限公司 | Image processing method, image processing device, electronic equipment and computer readable storage medium |
| CN114241127A (en) * | 2021-12-01 | 2022-03-25 | 维沃移动通信有限公司 | Panoramic image generation method and device, electronic equipment and medium |
| CN116740261A (en) * | 2022-03-02 | 2023-09-12 | 腾讯科技（深圳）有限公司 | Image reconstruction method and device and training method and device of image reconstruction model |
| CN114549731A (en) * | 2022-04-22 | 2022-05-27 | 清华大学 | Method and device for generating visual angle image, electronic equipment and storage medium |
| CN116977392A (en) * | 2022-12-29 | 2023-10-31 | 腾讯科技（深圳）有限公司 | Image generation method, device, electronic equipment and storage medium |
| CN116012432A (en) * | 2023-01-18 | 2023-04-25 | 广州南天电脑系统有限公司 | Stereoscopic panoramic image generation method and device and computer equipment |
| CN116958378A (en) * | 2023-02-21 | 2023-10-27 | 腾讯科技（深圳）有限公司 | Face map reconstruction method, device, computer readable medium and electronic equipment |
| CN116310036A (en) * | 2023-03-31 | 2023-06-23 | 北京字跳网络技术有限公司 | Scene rendering method, device, equipment, computer readable storage medium and product |
| CN116501209A (en) * | 2023-05-09 | 2023-07-28 | 网易（杭州）网络有限公司 | Editing view angle adjusting method and device, electronic equipment and readable storage medium |
| CN116310046A (en) * | 2023-05-16 | 2023-06-23 | 腾讯科技（深圳）有限公司 | Image processing method, device, computer and storage medium |
| CN116977531A (en) * | 2023-07-28 | 2023-10-31 | 腾讯科技（深圳）有限公司 | Three-dimensional texture image generation method, three-dimensional texture image generation device, computer equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagation;Bangbang Yang et al;arxiv;20231020;第1-14页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117197319A (en) | 2023-12-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11538229B2 (en) | Image processing method and apparatus, electronic device, and computer-readable storage medium | |
| Wheeler et al. | Virtual interaction and visualisation of 3D medical imaging data with VTK and Unity | |
| US9626790B1 (en) | View-dependent textures for interactive geographic information system | |
| EP2643820B1 (en) | Rendering and navigating photographic panoramas with depth information in a geographic information system | |
| US20130321396A1 (en) | Multi-input free viewpoint video processing pipeline | |
| CN112560137A (en) | Multi-model fusion method and system based on smart city | |
| US20110242271A1 (en) | Synthesizing Panoramic Three-Dimensional Images | |
| CN111142967B (en) | Augmented reality display method and device, electronic equipment and storage medium | |
| Wei et al. | Object-based illumination estimation with rendering-aware neural networks | |
| CN116097316A (en) | Object recognition neural network for modeless central prediction | |
| CN114531553B (en) | Method, device, electronic equipment and storage medium for generating special effect video | |
| CN114926612A (en) | Aerial panoramic image processing and immersive display system | |
| Pintore et al. | Instant automatic emptying of panoramic indoor scenes | |
| CN112288878B (en) | Augmented reality preview method and preview device, electronic equipment and storage medium | |
| Cui et al. | Fusing surveillance videos and three‐dimensional scene: A mixed reality system | |
| US20230260218A1 (en) | Method and apparatus for presenting object annotation information, electronic device, and storage medium | |
| KR102551914B1 (en) | Method and system for generating interactive object viewer | |
| CN117197319B (en) | Image generation method, device, electronic equipment and storage medium | |
| CN116543086A (en) | Nerve radiation field processing method and device and electronic equipment | |
| EP4283566A2 (en) | Single image 3d photography with soft-layering and depth-aware inpainting | |
| CN110689609B (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN112884888A (en) | Exhibition display method, system, equipment and medium based on mixed reality | |
| Kolivand et al. | Livephantom: Retrieving virtual world light data to real environments | |
| CN113298868B (en) | Model building method, device, electronic equipment, medium and program product | |
| CN112184873B (en) | Fractal graph creation method, fractal graph creation device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |