CN114792354B - Model processing method and device, storage medium and electronic equipment - Google Patents
Model processing method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN114792354B CN114792354B CN202210712217.XA CN202210712217A CN114792354B CN 114792354 B CN114792354 B CN 114792354B CN 202210712217 A CN202210712217 A CN 202210712217A CN 114792354 B CN114792354 B CN 114792354B
- Authority
- CN
- China
- Prior art keywords
- texture
- preset
- target
- model
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
- G06T2219/20—Indexing scheme for editing of 3D models
- G06T2219/2016—Rotation, translation, scaling
Abstract
The model processing method can acquire two-dimensional images of an appointed three-dimensional model of a target posture at a plurality of preset visual angles by rotating the preset three-dimensional model, determine a target material structure model corresponding to a chartlet texture in the preset three-dimensional model according to the two-dimensional images, effectively improve the accuracy of an extraction result of the target material structure model, and effectively improve the extraction efficiency due to the fact that manual one-by-one labeling is not needed.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to a model processing method and apparatus, a storage medium, and an electronic device.
Background
In three-dimensional rendering, in order to enhance the display effect of a building, regions such as window glass and a curtain wall in a three-dimensional model need to be enhanced and rendered, so that the position of the window glass or the curtain wall needs to be extracted from a texture map of the three-dimensional model, and the traditional method is that either manual operation is used for manually marking the regions of the window glass and the curtain wall in the texture map and then the regions related to a three-dimensional rendering system are informed of effect enhancement; or the window glass and the curtain wall area in the texture map are extracted by a traditional image analysis method, however, for a three-dimensional model with more window glass areas and more complex curtain wall distribution in a building, the positions of the window glass or the curtain wall cannot be efficiently and accurately extracted no matter manual operation or traditional image recognition is adopted.
Disclosure of Invention
The purpose of the disclosure is to provide a model processing method, a model processing device, a storage medium and an electronic device.
In order to achieve the above object, a first aspect of the present disclosure provides a model processing method, including:
acquiring a directional bounding box corresponding to a preset three-dimensional model, wherein the preset three-dimensional model comprises a mapping texture for representing a target material structure;
rotating the preset three-dimensional model, and stopping rotating the preset three-dimensional model under the condition that any surface of the direction bounding box is parallel to a specified coordinate axis in a Cartesian coordinate system to obtain a specified three-dimensional model of a target posture;
shooting the appointed three-dimensional model at a plurality of preset visual angles to obtain a two-dimensional image corresponding to each preset visual angle;
and determining a target material structure model corresponding to the mapping texture in the preset three-dimensional model according to the plurality of two-dimensional images corresponding to the plurality of preset visual angles.
Optionally, the determining, according to the plurality of two-dimensional images corresponding to the plurality of preset viewing angles, a target material structure model corresponding to the chartlet texture in the preset three-dimensional model includes:
acquiring a first target area corresponding to the texture of the map in each two-dimensional image through a first preset deep learning model;
performing image fusion on the first target areas in the two-dimensional images corresponding to the preset visual angles to obtain a position area of the chartlet texture in the appointed three-dimensional model;
acquiring a texture image on the position area in the specified three-dimensional model;
and determining a target material structure model corresponding to the chartlet texture according to the texture image.
Optionally, the obtaining, by the first preset deep learning model, a first target region corresponding to the chartlet texture in each of the two-dimensional images includes:
inputting the two-dimensional image into the down-sampling module to obtain a first feature map output by the down-sampling module, wherein the first feature map is used for describing global features of the two-dimensional image;
and inputting the first feature map into the first texture recognition module so that the first texture recognition module outputs a first target area corresponding to the chartlet texture in the two-dimensional image.
Optionally, the determining, according to the texture image, a target material structure model corresponding to the map texture includes:
inputting the texture image into a second preset deep learning model to obtain a second target area corresponding to the chartlet texture in the texture image;
acquiring a target texture image of the second target area;
and determining a target material structure model corresponding to the map texture according to the target texture image.
Optionally, the second preset deep learning model includes an upsampling module and a second texture recognition module, and the inputting the texture image into the second preset deep learning model to obtain a second target region corresponding to the texture of the map in the texture image includes:
inputting the texture image into the up-sampling module to obtain a second feature map output by the up-sampling module, wherein the second feature map is used for describing the refined features of the map texture in the texture image;
inputting the second feature map into the second texture recognition module to enable the second texture recognition module to output a second target region of the map texture in the texture image.
Optionally, the determining, according to the target texture image, a target material structure model corresponding to the map texture includes:
performing edge detection on the target texture image to obtain a set of to-be-determined boundary lines corresponding to the texture of the map;
determining a target boundary line of the chartlet texture from the set of boundary lines to be determined by using a flooding algorithm;
and determining a target material structure model corresponding to the chartlet texture in the specified three-dimensional model according to the target boundary line.
Optionally, the method further comprises:
and performing projection transformation on the target material structure model to determine a target texture area of the map texture.
A second aspect of the present disclosure provides a model processing apparatus, the apparatus including:
the acquisition module is configured to acquire a directional bounding box corresponding to a preset three-dimensional model, wherein the preset three-dimensional model comprises a mapping texture used for representing a target material structure;
a first determination module configured to rotate the preset three-dimensional model, and stop rotating the preset three-dimensional model when any surface of the orientation bounding box is parallel to a designated coordinate axis in a Cartesian coordinate system, so as to obtain a designated three-dimensional model of a target pose;
a second determining module configured to capture the specified three-dimensional model at a plurality of preset viewing angles to obtain a two-dimensional image corresponding to each of the preset viewing angles;
a third determining module configured to determine, according to the plurality of two-dimensional images corresponding to the plurality of preset viewing angles, a target material structure model corresponding to the map texture in the preset three-dimensional model.
Optionally, the third determining module is configured to:
acquiring a first target area corresponding to the chartlet texture in each two-dimensional image through a first preset deep learning model;
performing image fusion on the first target areas in the two-dimensional images corresponding to the preset visual angles to obtain a position area of the chartlet texture in the designated three-dimensional model;
acquiring a texture image on the position area in the specified three-dimensional model;
and determining a target material structure model corresponding to the chartlet texture according to the texture image.
Optionally, the first preset deep learning model includes a down-sampling module and a first texture recognition module, and the third determining module is configured to:
inputting the two-dimensional image into the down-sampling module to obtain a first feature map output by the down-sampling module, wherein the first feature map is used for describing global features of the two-dimensional image;
and inputting the first feature map into the first texture recognition module so that the first texture recognition module outputs a first target area corresponding to the chartlet texture in the two-dimensional image.
Optionally, the third determining module is configured to:
inputting the texture image into a second preset deep learning model to obtain a second target area corresponding to the chartlet texture in the texture image;
acquiring a target texture image of the second target area;
and determining a target material structure model corresponding to the mapping texture according to the target texture image.
Optionally, the second preset deep learning model comprises an upsampling module and a second texture recognition module, and the third determining module is configured to:
inputting the texture image into the up-sampling module to obtain a second feature map output by the up-sampling module, wherein the second feature map is used for describing the refined features of the map texture in the texture image;
inputting the second feature map into the second texture recognition module to enable the second texture recognition module to output a second target region of the map texture in the texture image.
Optionally, the third determining module is configured to:
performing edge detection on the target texture image to obtain a set of undetermined boundary lines corresponding to the chartlet texture;
determining a target boundary line of the chartlet texture from the set of boundary lines to be determined by using a flooding algorithm;
and determining a target material structure model corresponding to the chartlet texture in the specified three-dimensional model according to the target boundary line.
Optionally, the apparatus further comprises:
a fourth determination module configured to perform projective transformation on the target material structure model to determine a target texture region of the map texture.
A third aspect of the disclosure provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of the first aspect described above.
A fourth aspect of the present disclosure provides an electronic device, comprising:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to carry out the steps of the method of the first aspect above.
According to the technical scheme, a direction bounding box corresponding to a preset three-dimensional model is obtained, wherein the preset three-dimensional model comprises a mapping texture used for representing a target material structure; rotating the preset three-dimensional model, and stopping rotating the preset three-dimensional model under the condition that any surface of the direction bounding box is parallel to a specified coordinate axis in a Cartesian coordinate system to obtain a specified three-dimensional model of a target posture; shooting the appointed three-dimensional model at a plurality of preset visual angles to obtain a two-dimensional image corresponding to each preset visual angle; determining a target material structure model corresponding to the chartlet texture in the preset three-dimensional model according to a plurality of two-dimensional images corresponding to the plurality of preset visual angles; therefore, two-dimensional images of the appointed three-dimensional model of the target posture at a plurality of preset visual angles can be obtained by rotating the preset three-dimensional model, the target material structure model corresponding to the chartlet texture in the preset three-dimensional model is determined according to the two-dimensional images, the target material structure model in the preset three-dimensional model can be efficiently and accurately extracted, and therefore reliable data basis can be provided for the rendering of the target material structure model.
Additional features and advantages of the present disclosure will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:
FIG. 1 is a flow chart diagram of a method of model processing shown in an exemplary embodiment of the present disclosure;
FIG. 2 is a flow diagram illustrating a method of model processing according to the embodiment shown in FIG. 1;
FIG. 3 is a flow diagram illustrating a method of model processing according to the embodiment shown in FIG. 2;
FIG. 4 is a block diagram of a model processing device, shown in an exemplary embodiment of the present disclosure;
FIG. 5 is a block diagram of an electronic device shown in accordance with an example embodiment.
Detailed Description
The following detailed description of the embodiments of the disclosure refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present disclosure, are given by way of illustration and explanation only, not limitation.
It should be noted that all actions of acquiring signals, information or data in the present disclosure are performed under the premise of complying with the corresponding data protection regulation policy of the country of the location and obtaining the authorization given by the owner of the corresponding device.
Before describing the embodiments of the present disclosure in detail, first, the following description is made on an application scenario of the present disclosure, and the present disclosure may be applied to a process of identifying a target material structure on a three-dimensional building model, and after identifying the target material structure, the target material structure may be enhanced and rendered, where the target material structure may be glass material such as door and window glass, curtain wall, and the like, or may be other specified material such as wood, steel, soil, and the like. Taking a target material structure made of glass as an example, in the related art, for a three-dimensional model with more window glass areas and more complex curtain wall distribution in a building, the problems of low efficiency and poor accuracy of the identification result of identifying the position of the target material structure generally exist.
In order to solve the technical problems, the present disclosure provides a model processing method, an apparatus, a storage medium, and an electronic device, where the model processing method can obtain two-dimensional images of an appointed three-dimensional model of a target pose at a plurality of preset viewing angles by rotating a preset three-dimensional model, and determine a target material structure model corresponding to a chartlet texture in the preset three-dimensional model according to the plurality of two-dimensional images, so that accuracy of an extraction result of the target material structure model can be effectively improved, and extraction efficiency can also be effectively improved because manual one-by-one labeling is not required.
The technical scheme of the disclosure is explained in detail by combining specific embodiments.
FIG. 1 is a flow chart of a model processing method shown in an exemplary embodiment of the present disclosure; as shown in fig. 1, the method may include:
The preset three-dimensional model can be a three-dimensional building model, and the target material structure can be an object structure made of a specified material, such as a door, a window, a curtain wall and the like made of glass, and can also be a wooden (iron) door, a window, a picture frame, a billboard and the like. The texture of the map is the texture of the target material structure on the three-dimensional model. It should be noted that, the technical solution for obtaining the directional bounding box of the three-dimensional model is mature in the prior art, and the disclosure does not limit this.
And 102, rotating the preset three-dimensional model, and stopping rotating the preset three-dimensional model under the condition that any surface of the direction bounding box is parallel to a specified coordinate axis in a Cartesian coordinate system to obtain the specified three-dimensional model of the target posture.
The designated coordinate axis may be any one of an x axis, a y axis and a z axis, and the directional bounding box is a rectangular parallelepiped and generally includes six faces, i.e., an upper face, a lower face, a left face, a right face, a front face and a rear face.
For example, when the predetermined three-dimensional model is a three-dimensional building model, if the ground of the three-dimensional building model coincides with (or is parallel to) the XOY plane in the coordinate system, the predetermined three-dimensional model may be rotated around the Z axis so that one plane of the directional bounding box corresponding to the three-dimensional building model is parallel to the X axis or the Y axis.
The shooting with the plurality of preset viewing angles may include shooting right to the front, back, left, and right of the bounding box in the direction corresponding to the designated three-dimensional model, and may further include any shooting angle around the designated three-dimensional model, for example, shooting may be performed at a shooting angle of 45 degrees with respect to the ground where the designated three-dimensional model is located, and shooting may be performed once around the designated three-dimensional model every 15 degrees, 20 degrees, or 30 degrees, so as to obtain the two-dimensional image, that is, shooting may be performed once around the three-dimensional model at 45 degrees above the designated three-dimensional model.
And step 104, determining a target material structure model corresponding to the chartlet texture in the preset three-dimensional model according to the two-dimensional images corresponding to the preset visual angles.
The target material structure model is a target material structure part in the specified three-dimensional model, namely a three-dimensional model corresponding to the target material structure.
In this step, the texture region position of the target material structure in each two-dimensional image can be identified by training the target image identification model, and then three-dimensional reconstruction is performed according to a plurality of texture region positions corresponding to a plurality of two-dimensional images, so as to obtain the target material structure model identified by the target. The target image recognition model can be an image recognition model based on a deep neural network algorithm, and the technology for training the image recognition model based on the deep neural network algorithm according to the recognition requirement is mature in the prior art, and is not described in detail in the disclosure.
According to the technical scheme, the two-dimensional images of the appointed three-dimensional model of the target posture at the plurality of preset visual angles can be obtained by rotating the preset three-dimensional model, the target material structure model corresponding to the chartlet texture in the preset three-dimensional model is determined according to the plurality of two-dimensional images, the target material structure model in the preset three-dimensional model can be efficiently and accurately extracted, and therefore reliable data basis can be provided for the rendering of the target material structure model.
FIG. 2 is a flow diagram illustrating a method of model processing according to the embodiment shown in FIG. 1; as shown in fig. 2, the determining the target material structure model corresponding to the mapping texture in the predetermined three-dimensional model according to the two-dimensional images corresponding to the predetermined viewing angles in step 104 in fig. 1 may include the following steps:
In this step, the two-dimensional image may be input to the downsampling module to obtain a first feature map output by the downsampling module, where the first feature map is used to describe a global feature of the two-dimensional image; and inputting the first feature map into the first texture recognition module so that the first texture recognition module outputs a first target area corresponding to the chartlet texture in the two-dimensional image.
It should be noted that the first texture identifying module may at least include a classification layer, where the classification layer is configured to determine, according to the first feature map, a probability that each pixel in the two-dimensional image belongs to the map texture, and use a pixel, of which the probability that each pixel belongs to the map texture is greater than a preset threshold, as a pixel of the first target region, so as to obtain the first target region. The training process of the first preset deep learning model may be to perform model training on a first preset initial model through a first training data set to obtain the first preset deep learning model, the first preset initial model includes an initial down-sampling module, the first training data set includes label data for texture structure categories, for example, when identifying a glass material structure, the label data may be a glass material structure.
In this step, a first target region in the plurality of two-dimensional images corresponding to the plurality of preset viewing angles may be subjected to projection transformation according to the camera parameter and the depth map of each two-dimensional image, so as to obtain a position region of a target material structure corresponding to the texture of the map in the specified three-dimensional model.
It should be noted that the above image fusion may be understood as three-dimensional reconstruction, and performing three-dimensional reconstruction by using the projection transformation method according to two-dimensional images under multiple viewing angles belongs to a mature technology in the art, and specific details of projection transformation may refer to a three-dimensional reconstruction process in the prior art, which is not described herein again.
In this step, a pixel value on each position coordinate in the position area may be obtained to obtain a texture image on the position area.
The step may include the steps shown in fig. 3, where fig. 3 is a flowchart of a model processing method according to the embodiment shown in fig. 2, and the step 1044 of determining the target material structure model corresponding to the chartlet texture according to the texture image shown in fig. 2 may include:
s1, inputting the texture image into a second preset deep learning model to obtain a second target area corresponding to the chartlet texture in the texture image.
In one possible implementation, the second preset deep learning model does not include a down-sampling module and an up-sampling module, that is, the texture image is not down-sampled or up-sampled, and the detection of the second target region where the texture is mapped is directly performed according to the feature data of the original image size of the texture image.
In another possible implementation, the second preset deep learning model includes an up-sampling module and a second texture recognition module, and an output of the up-sampling module is coupled to an input of the second texture recognition module.
In this step, the texture image may be input to the upsampling module to obtain a second feature map output by the upsampling module, where the second feature map is used to describe a fine feature of the texture of the map in the texture image; inputting the second feature map into the second texture recognition module, so that the second texture recognition module outputs a second target region of the mapped texture in the texture image.
It should be noted that the second texture identifying module may at least include a classification layer, where the classification layer is configured to determine, according to the second feature map, a probability that each pixel in the texture image belongs to the map texture, and use a pixel, of which the probability that each pixel belongs to the map texture is greater than a preset threshold, as a pixel of the second target region, so as to obtain the second target region. The training process of the second preset deep learning model may be model training of a second preset initial model through a second training data set to obtain the second preset deep learning model, the second preset initial model includes an initial up-sampling module, the second training data set includes label data of a texture corresponding to a structure category in an image, for example, when identifying a glass material structure, the label data may be a glass material structure. Because this first predetermined degree of deep learning model need acquire more global characteristics, consequently need to predetermine the two-dimensional image that the visual angle was gathered to many and down sample, however, because down sample can run off more image details, avoid the relatively poor problem of position testing result accuracy that the detail runs off and cause in this disclosure, predetermine degree of deep learning model through this second in this step and discern texture image, do not include down sample module in this second predetermined degree of deep learning model, can include the upsampling module, consequently, the save texture detail that can be more, be favorable to promoting the accuracy and the reliability of recognition result.
In addition, for the case that the texture of the preset three-dimensional model is formed by splicing a plurality of identical images (for example, the texture on 10 walls is completely identical, and the glass window on each wall is identical), a position region corresponding to the mapping texture of the target material structure (for example, the glass window) in each image can be obtained, and if the number of images having the glass window mapping texture in the position region in the plurality of identical images is greater than or equal to a preset threshold value, it is determined that the glass window mapping texture exists in the position region of each image, so that the glass window region which is not recognized by deep learning can be supplemented, and a more accurate second target region can be obtained.
Furthermore, when the texture of the map for the target material structure is arranged on the surface of the preset three-dimensional model according to a certain rule, whether a missing identification part exists or not can be determined according to the rule according to the position area identified as the glass window, and the missing identification part is supplemented when the missing identification part exists. For example, in a high-rise residential building, a plurality of glass windows are distributed according to a certain row-column rule (for example, a rectangular glass window is arranged every 3 meters from bottom to top, and the shapes and sizes of all the rectangular glass windows are the same), when a glass window area in the high-rise residential building is identified, if a row (or a column) of the identified glass windows after a certain rectangular glass window is found to miss several pixels, the missed identified several pixels can be supplemented, that is, the missed identified several pixels are also used as pixels in a second target area, so that a supplemented second target area is obtained.
And S2, acquiring a target texture image of the second target area.
The second target area is a refined area where the texture of the target material structure corresponding to the map is located more accurately, and the second target area is obtained through the second preset deep learning model.
And S3, determining a target material structure model corresponding to the texture of the map according to the target texture image.
In this step, a possible implementation manner is to perform three-dimensional reconstruction according to the target texture image to obtain the target material structure model.
Another possible implementation manner is that edge detection is performed on the target texture image to obtain a set of to-be-determined boundary lines corresponding to the texture of the map; determining a target boundary line of the chartlet texture from the set of the boundary lines to be determined by using a flooding algorithm; and determining a target material structure model corresponding to the mapping texture in the specified three-dimensional model according to the target boundary line.
The boundary line can be extracted through DexiNet (Dense sensing Network for edge detection), then the target boundary line of the chartlet texture is determined from the undetermined boundary line set according to the characteristic that the color difference of the target material structure is small by using a flood algorithm, and a three-dimensional model (3 DPolygon, three-dimensional polygon) with an accurate target material structure is obtained, so that the target boundary line of the target material structure can be determined more finely in a pixel dimension, and the accuracy of a boundary line identification result can be effectively guaranteed.
It should be noted that, when the target material structure is a curtain wall, the set of undetermined boundary lines further includes a boundary of a keel line, and the generated target material structure model includes a keel line model. In addition, it should be further noted that the above DexiNet method and the flood algorithm are common in the field of image processing, and specific calculation processes can be referred to related descriptions in the prior art, which is not limited in the present disclosure.
According to the technical scheme, the accurate region of the chartlet texture in the texture image can be obtained through the second preset deep learning model, and then the target boundary line of the target material structure can be determined more finely in the pixel dimension through an edge detection method and a flooding algorithm, so that the accuracy of the boundary line identification result can be effectively guaranteed.
Optionally, the method may further include:
and performing projection transformation on the target material structure model to determine a target texture area of the mapping texture.
It should be noted that, after the target material structure portion in the specified three-dimensional model is obtained, a target texture region corresponding to the chartlet texture belonging to the target material structure may be determined in the texture image, so as to provide a reliable data basis for the rendering of the preset three-dimensional model in the later period.
According to the technical scheme, the two-dimensional images of the appointed three-dimensional model of the target posture at the plurality of preset visual angles can be obtained by rotating the preset three-dimensional model, the target material structure model corresponding to the chartlet texture in the preset three-dimensional model is determined according to the plurality of two-dimensional images, the target material structure model in the preset three-dimensional model can be efficiently and accurately extracted, and therefore reliable data basis can be provided for the rendering of the target material structure model.
FIG. 4 is a block diagram of a model processing device, shown in an exemplary embodiment of the present disclosure; as shown in fig. 4, the model processing means may include:
an obtaining module 401 configured to obtain a directional bounding box corresponding to a preset three-dimensional model, where the preset three-dimensional model includes a mapping texture for representing a target material structure;
a first determining module 402 configured to rotate the preset three-dimensional model, and stop rotating the preset three-dimensional model if any surface of the orientation bounding box is parallel to a designated coordinate axis in a cartesian coordinate system, so as to obtain a designated three-dimensional model of a target pose;
a second determining module 403, configured to capture the specified three-dimensional model at a plurality of preset viewing angles to obtain a two-dimensional image corresponding to each of the preset viewing angles;
a third determining module 404, configured to determine a target material structure model corresponding to the texture of the map in the predetermined three-dimensional model according to the two-dimensional images corresponding to the predetermined viewing angles.
According to the technical scheme, the two-dimensional images of the appointed three-dimensional model of the target posture at the plurality of preset visual angles can be obtained by rotating the preset three-dimensional model, the target material structure model corresponding to the chartlet texture in the preset three-dimensional model is determined according to the plurality of two-dimensional images, the target material structure model in the preset three-dimensional model can be efficiently and accurately extracted, and therefore reliable data basis can be provided for the rendering of the target material structure model.
Optionally, the third determining module 404 is configured to:
acquiring a first target area corresponding to the texture of the map in each two-dimensional image through a first preset deep learning model;
performing image fusion on the first target areas in the two-dimensional images corresponding to the preset visual angles to obtain a position area of the chartlet texture in the designated three-dimensional model;
acquiring a texture image on the position area in the specified three-dimensional model;
and determining a target material structure model corresponding to the mapping texture according to the texture image.
Optionally, the first preset deep learning model includes a downsampling module and a first texture identifying module, and the third determining module 404 is configured to:
inputting the two-dimensional image into the down-sampling module to obtain a first feature map output by the down-sampling module, wherein the first feature map is used for describing the global features of the two-dimensional image;
and inputting the first feature map into the first texture recognition module so that the first texture recognition module outputs a first target area corresponding to the chartlet texture in the two-dimensional image.
Optionally, the third determining module 404 is configured to:
inputting the texture image into a second preset deep learning model to obtain a second target area corresponding to the mapping texture in the texture image;
acquiring a target texture image of the second target area;
and determining a target material structure model corresponding to the mapping texture according to the target texture image.
Optionally, the second preset deep learning model includes an upsampling module and a second texture recognition module, and the third determining module 404 is configured to:
inputting the texture image into the up-sampling module to obtain a second feature map output by the up-sampling module, wherein the second feature map is used for describing a refined feature of the chartlet texture in the texture image;
inputting the second feature map into the second texture recognition module, so that the second texture recognition module outputs a second target region of the mapped texture in the texture image.
Optionally, the third determining module 404 is configured to:
performing edge detection on the target texture image to obtain a set of undetermined boundary lines corresponding to the texture of the map;
determining a target boundary line of the chartlet texture from the set of the boundary lines to be determined by using a flooding algorithm;
and determining a target material structure model corresponding to the mapping texture in the specified three-dimensional model according to the target boundary line.
According to the technical scheme, the accurate area of the texture of the map in the texture image can be obtained through the second preset deep learning model, and then the target boundary line of the target material structure can be determined more finely in the pixel dimension through an edge detection method and a flooding algorithm, so that the accuracy of the boundary line recognition result can be effectively guaranteed.
Optionally, the apparatus further comprises:
a fourth determining module 405 configured to perform projective transformation on the target material structure model to determine a target texture region of the map texture.
According to the technical scheme, the two-dimensional images of the appointed three-dimensional model of the target posture at a plurality of preset visual angles can be obtained by rotating the preset three-dimensional model, the target material structure model corresponding to the chartlet texture in the preset three-dimensional model is determined according to the two-dimensional images, the target material structure model in the preset three-dimensional model can be efficiently and accurately extracted, and therefore reliable data basis can be provided for the rendering of the target material structure model.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
FIG. 5 is a block diagram illustrating an electronic device in accordance with an example embodiment. As shown in fig. 5, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input/output interface 704, and a communication component 705.
The processor 701 is configured to control the overall operation of the electronic device 700, so as to complete all or part of the steps of the model processing method. The memory 702 is used to store various types of data to support operation at the electronic device 700, such as instructions for any application or method operating on the electronic device 700 and application-related data, such as contact data, transmitted and received messages, pictures, audio, video, and so forth. The Memory 702 may be implemented by any type of volatile or non-volatile Memory device or combination thereof, such as Static Random Access Memory (SRAM), electrically Erasable Programmable Read-Only Memory (EEPROM), erasable Programmable Read-Only Memory (EPROM), programmable Read-Only Memory (PROM), read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia components 703 may include screen and audio components. Wherein the screen may be, for example, a touch screen and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may further be stored in the memory 702 or transmitted through the communication component 705. The audio assembly further comprises at least one speaker for outputting audio signals. The input/output interface 704 provides an interface between the processor 701 and other interface modules, such as a keyboard, mouse, buttons, and the like. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless Communication, such as Wi-Fi, bluetooth, near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, or combinations thereof, which is not limited herein. The corresponding communication component 705 may thus include: wi-Fi module, bluetooth module, NFC module, etc.
In an exemplary embodiment, the electronic Device 700 may be implemented by one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described model Processing method.
In another exemplary embodiment, there is also provided a computer readable storage medium including program instructions, which when executed by a processor, implement the steps of the model processing method described above. For example, the computer readable storage medium may be the memory 702 described above including program instructions that are executable by the processor 701 of the electronic device 700 to perform the model processing method described above.
The preferred embodiments of the present disclosure are described in detail above with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details in the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.
It should be noted that, in the foregoing embodiments, various features described in the above embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various combinations that are possible in the present disclosure are not described again.
In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure as long as it does not depart from the gist of the present disclosure.
Claims (8)
1. A method of model processing, the method comprising:
acquiring a directional bounding box corresponding to a preset three-dimensional model, wherein the preset three-dimensional model comprises a mapping texture for representing a target material structure;
rotating the preset three-dimensional model, and under the condition that any surface in the direction bounding box is parallel to a specified coordinate axis in a Cartesian coordinate system, stopping rotating the preset three-dimensional model to obtain a specified three-dimensional model of a target posture;
shooting the appointed three-dimensional model at a plurality of preset visual angles to obtain a two-dimensional image corresponding to each preset visual angle, wherein the shooting of the appointed three-dimensional model at the plurality of preset visual angles comprises shooting a circle around the appointed three-dimensional model at 45 degrees above the appointed three-dimensional model in an oblique mode;
determining a target material structure model corresponding to the chartlet texture in the preset three-dimensional model according to a plurality of two-dimensional images corresponding to the plurality of preset visual angles;
wherein, the determining a target material structure model corresponding to the texture of the map in the preset three-dimensional model according to the plurality of two-dimensional images corresponding to the plurality of preset viewing angles comprises:
acquiring a first target area corresponding to the texture of the map in each two-dimensional image through a first preset deep learning model;
performing image fusion on the first target areas in the two-dimensional images corresponding to the preset visual angles to obtain a position area of the chartlet texture in the designated three-dimensional model;
acquiring a texture image on the position area in the specified three-dimensional model;
determining a target material structure model corresponding to the chartlet texture according to the texture image;
the determining a target material structure model corresponding to the chartlet texture according to the texture image includes:
inputting the texture image into a second preset deep learning model to obtain a second target area corresponding to the chartlet texture in the texture image;
acquiring a position area corresponding to the mapping texture of the target material structure in each image under the condition that the texture of the preset three-dimensional model is formed by splicing a plurality of images with the same mapping texture, and determining that the mapping texture of the target material structure exists in the appointed position area in each image to obtain a more accurate second target area if the number of the images with the mapping texture of the target material structure in the appointed position area is greater than or equal to a preset threshold value;
aiming at the situation that the texture of the map of the target material structure is arranged on the surface of the preset three-dimensional model according to a preset rule, determining whether a part which is not identified exists according to the preset rule according to the position area which is identified as the target material structure, and supplementing the part which is not identified so as to obtain a second target area after the supplement under the condition that the part which is not identified exists;
acquiring a target texture image of the second target area;
and determining a target material structure model corresponding to the mapping texture according to the target texture image.
2. The method according to claim 1, wherein the first preset deep learning model includes a down-sampling module and a first texture recognition module, and the obtaining a first target region corresponding to the chartlet texture in each of the two-dimensional images through the first preset deep learning model includes:
inputting the two-dimensional image into the down-sampling module to obtain a first feature map output by the down-sampling module, wherein the first feature map is used for describing global features of the two-dimensional image;
and inputting the first feature map into the first texture recognition module so that the first texture recognition module outputs a first target area corresponding to the chartlet texture in the two-dimensional image.
3. The method according to claim 1, wherein the second preset deep learning model comprises an upsampling module and a second texture recognition module, and the inputting the texture image into the second preset deep learning model to obtain a second target region corresponding to the map texture in the texture image comprises:
inputting the texture image into the upsampling module to obtain a second feature map output by the upsampling module, wherein the second feature map is used for describing a refined feature of the chartlet texture in the texture image;
inputting the second feature map into the second texture recognition module to cause the second texture recognition module to output a second target region of the map texture in the texture image.
4. The method of claim 1, wherein determining the target material structure model corresponding to the texture of the map according to the target texture image comprises:
performing edge detection on the target texture image to obtain a set of undetermined boundary lines corresponding to the chartlet texture;
determining a target boundary line of the chartlet texture from the set of boundary lines to be determined by using a flooding algorithm;
and determining a target material structure model corresponding to the chartlet texture in the specified three-dimensional model according to the target boundary line.
5. The method according to any one of claims 1-4, further comprising:
and performing projection transformation on the target material structure model to determine a target texture area of the mapping texture.
6. A model processing apparatus, characterized in that the apparatus comprises:
the acquisition module is configured to acquire a directional bounding box corresponding to a preset three-dimensional model, wherein the preset three-dimensional model comprises a mapping texture used for representing a target material structure;
a first determination module configured to rotate the preset three-dimensional model, and stop rotating the preset three-dimensional model when any surface of the orientation bounding box is parallel to a designated coordinate axis in a Cartesian coordinate system, so as to obtain a designated three-dimensional model of a target pose;
a second determining module configured to capture the specified three-dimensional model at a plurality of preset viewing angles to obtain a two-dimensional image corresponding to each of the preset viewing angles, wherein the capturing of the specified three-dimensional model at the plurality of preset viewing angles includes capturing a circle around the specified three-dimensional model at 45 degrees obliquely above the specified three-dimensional model;
a third determining module, configured to determine, according to the plurality of two-dimensional images corresponding to the plurality of preset viewing angles, a target material structure model corresponding to the chartlet texture in the preset three-dimensional model;
wherein the third determination module is configured to:
acquiring a first target area corresponding to the texture of the map in each two-dimensional image through a first preset deep learning model;
performing image fusion on the first target areas in the two-dimensional images corresponding to the preset visual angles to obtain a position area of the chartlet texture in the designated three-dimensional model;
acquiring a texture image on the position area in the specified three-dimensional model;
determining a target material structure model corresponding to the chartlet texture according to the texture image;
the third determination module configured to:
inputting the texture image into a second preset deep learning model to obtain a second target area corresponding to the chartlet texture in the texture image;
acquiring a position area corresponding to the mapping texture of the target material structure in each image under the condition that the texture of the preset three-dimensional model is formed by splicing a plurality of images with the same mapping texture, and determining that the mapping texture of the target material structure exists in the appointed position area in each image to obtain a more accurate second target area if the number of the images with the mapping textures of the target material structure in the appointed position area is greater than or equal to a preset threshold value;
aiming at the situation that the texture of the map of the target material structure is arranged on the surface of the preset three-dimensional model according to a preset rule, determining whether a part which is not identified exists according to the preset rule according to the position area which is identified as the target material structure, and supplementing the part which is not identified so as to obtain a second target area after the supplement under the condition that the part which is not identified exists;
acquiring a target texture image of the second target area;
and determining a target material structure model corresponding to the mapping texture according to the target texture image.
7. A non-transitory computer readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 5.
8. An electronic device, comprising:
a memory having a computer program stored thereon;
a processor for executing the computer program in the memory to carry out the steps of the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210712217.XA CN114792354B (en) | 2022-06-22 | 2022-06-22 | Model processing method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210712217.XA CN114792354B (en) | 2022-06-22 | 2022-06-22 | Model processing method and device, storage medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114792354A CN114792354A (en) | 2022-07-26 |
| CN114792354B true CN114792354B (en) | 2022-11-11 |
Family
ID=82463687
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210712217.XA Active CN114792354B (en) | 2022-06-22 | 2022-06-22 | Model processing method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114792354B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107093205A (en) * | 2017-03-15 | 2017-08-25 | 北京航空航天大学 | A kind of three dimensions building window detection method for reconstructing based on unmanned plane image |
| CN113012293A (en) * | 2021-03-22 | 2021-06-22 | 平安科技（深圳）有限公司 | Stone carving model construction method, device, equipment and storage medium |
| CN114612641A (en) * | 2020-12-07 | 2022-06-10 | 阿里巴巴集团控股有限公司 | Material migration method and device and data processing method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105741348B (en) * | 2016-01-28 | 2018-06-12 | 北京航空航天大学 | A kind of threedimensional model edit methods of structure adaptive |
| CN111415389B (en) * | 2020-03-18 | 2023-08-29 | 清华大学 | Label-free six-dimensional object posture prediction method and device based on reinforcement learning |
| CN111402122A (en) * | 2020-03-20 | 2020-07-10 | 北京字节跳动网络技术有限公司 | Image mapping processing method and device, readable medium and electronic equipment |
-
2022
- 2022-06-22 CN CN202210712217.XA patent/CN114792354B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107093205A (en) * | 2017-03-15 | 2017-08-25 | 北京航空航天大学 | A kind of three dimensions building window detection method for reconstructing based on unmanned plane image |
| CN114612641A (en) * | 2020-12-07 | 2022-06-10 | 阿里巴巴集团控股有限公司 | Material migration method and device and data processing method |
| CN113012293A (en) * | 2021-03-22 | 2021-06-22 | 平安科技（深圳）有限公司 | Stone carving model construction method, device, equipment and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 基于浮雕纹理贴图的复杂模型快速显示;荣国栋等;《系统仿真学报》;20011230;正文第55-72页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114792354A (en) | 2022-07-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109887003B (en) | Method and equipment for carrying out three-dimensional tracking initialization | |
| CA3204361A1 (en) | Image processing method, image processing apparatus, and non-transitory storage medium | |
| CN109635783B (en) | Video monitoring method, device, terminal and medium | |
| US8831336B2 (en) | Method, system and computer program product for detecting an object in response to depth information | |
| WO2020206708A1 (en) | Obstacle recognition method and apparatus, computer device, and storage medium | |
| US20140168367A1 (en) | Calibrating visual sensors using homography operators | |
| CN108765333B (en) | Depth map perfecting method based on depth convolution neural network | |
| CN109711246B (en) | Dynamic object recognition method, computer device and readable storage medium | |
| CN112669344A (en) | Method and device for positioning moving object, electronic equipment and storage medium | |
| CN113689578B (en) | Human body data set generation method and device | |
| CN109951635A (en) | It takes pictures processing method, device, mobile terminal and storage medium | |
| US20240029303A1 (en) | Three-dimensional target detection method and apparatus | |
| CN110866497A (en) | Robot positioning and image building method and device based on dotted line feature fusion | |
| WO2018176929A1 (en) | Image background blurring method and apparatus | |
| CN111340960B (en) | Image modeling method and device, storage medium and electronic equipment | |
| CN114662587A (en) | Three-dimensional target sensing method, device and system based on laser radar | |
| US20210304411A1 (en) | Map construction method, apparatus, storage medium and electronic device | |
| CN115984268B (en) | Target detection method and device based on machine vision, electronic equipment and medium | |
| CN114792354B (en) | Model processing method and device, storage medium and electronic equipment | |
| CN113486941B (en) | Live image training sample generation method, model training method and electronic equipment | |
| CN115345895A (en) | Image segmentation method and device for visual detection, computer equipment and medium | |
| CN115511870A (en) | Object detection method and device, electronic equipment and storage medium | |
| EP4148379A1 (en) | Visual positioning method and apparatus | |
| US11205064B1 (en) | Measuring quality of depth images in real time | |
| CN113744361A (en) | Three-dimensional high-precision map construction method and device based on trinocular vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address |
Address after: Room Y579, 3rd Floor, Building 3, No. 9 Keyuan Road, Daxing District Economic Development Zone, Beijing 102600 Patentee after: Beijing Feidu Technology Co.,Ltd. Address before: 100162 608, floor 6, building 1, courtyard 15, Xinya street, Daxing District, Beijing Patentee before: Beijing Feidu Technology Co.,Ltd. |
|
| CP03 | Change of name, title or address |