CN112950759B

CN112950759B - Three-dimensional house model construction method and device based on house panoramic image

Info

Publication number: CN112950759B
Application number: CN202110117065.4A
Authority: CN
Inventors: 杨永林; 白杰
Original assignee: Seashell Housing Beijing Technology Co Ltd
Current assignee: Seashell Housing Beijing Technology Co Ltd
Priority date: 2021-01-28
Filing date: 2021-01-28
Publication date: 2022-12-06
Anticipated expiration: 2041-01-28
Also published as: CN112950759A

Abstract

The embodiment of the disclosure discloses a three-dimensional house model construction method and device based on a house panoramic image. The method comprises the following steps: generating an initial three-dimensional house model by using the house panorama; carrying out article identification on the house panoramic image to obtain an article identification result; according to the item identification result, generating N three-dimensional item models corresponding to N specified items in the house panoramic image in the initial three-dimensional house model; wherein N is an integer greater than or equal to 1; according to the house panoramic image, respectively carrying out mapping processing on the N three-dimensional article models; and obtaining a target three-dimensional house model according to the initial three-dimensional house model after the N three-dimensional article models are subjected to mapping processing. Compared with the prior art, the three-dimensional house model constructed based on the house panorama in the embodiment of the disclosure can embody richer information, and the display effect is more visual and real, so that the model display effect and the user experience can be effectively improved.

Description

Three-dimensional house model construction method and device based on house panoramic image

Technical Field

The disclosure relates to the technical field of three-dimensional modeling, in particular to a three-dimensional house model construction method and device based on a house panorama.

Background

In some cases, a house panorama is needed to construct a three-dimensional house model, but the three-dimensional house model constructed by the current technology can only reflect the spatial structure information of a real house generally, but cannot reflect information of other aspects at all, so that the three-dimensional house model constructed by the current technology can reflect less information, and the model display effect and the user experience are poor.

Disclosure of Invention

The present disclosure is proposed to solve the above technical problems. The embodiment of the disclosure provides a three-dimensional house model construction method and device based on a house panorama.

According to an aspect of the embodiment of the present disclosure, a method for building a three-dimensional house model based on a house panorama is provided, including:

generating an initial three-dimensional house model by using a house panoramic picture;

carrying out article identification on the house panoramic image to obtain an article identification result;

according to the item identification result, generating N three-dimensional item models corresponding to N designated items in the house panoramic image in the initial three-dimensional house model; wherein N is an integer greater than or equal to 1;

respectively carrying out mapping processing on the N three-dimensional article models by using the house panoramic image;

and obtaining a target three-dimensional house model according to the initial three-dimensional house model after the N three-dimensional article models are subjected to mapping processing.

In an optional example, the generating, in the initial three-dimensional house model according to the item identification result, N three-dimensional item models corresponding to N designated items in the house panorama includes:

determining M reference points of a target specified item in the house panoramic image according to the item identification result; the target specified article is any one of N specified articles in the house panoramic image, and M is an integer greater than or equal to 3;

mapping the M fiducial points to the initial three-dimensional house model to determine M mapping points in the initial three-dimensional house model;

determining a stretching datum plane in the initial three-dimensional house model according to the M mapping points;

determining a stretching parameter corresponding to the target specified article;

and according to the stretching parameters corresponding to the target specified article, stretching the stretching datum plane to generate a three-dimensional article model corresponding to the target specified article in the initial three-dimensional house model.

In an optional example, the obtaining a target three-dimensional house model according to the initial three-dimensional house model after the mapping processing of the N three-dimensional article models comprises:

determining respective fragment parts of the N three-dimensional article models subjected to mapping processing;

and deleting the determined fragment part from the initial three-dimensional house model after the mapping processing is carried out on the N three-dimensional article models to obtain a target three-dimensional house model.

In one optional example, the stretch parameters include a stretch direction and a stretch distance;

the determining the fragment parts of the N three-dimensional article models subjected to mapping processing comprises the following steps:

determining a target plane; wherein the stretching direction is a direction pointing from the stretching reference plane to the target plane, and the target plane is parallel to the stretching reference plane and is at the stretching distance from the stretching reference plane;

determining a surface of the three-dimensional article model corresponding to the target designated article, which is opposite to the stretching datum plane;

and detecting the projection part of the determined surface relative to the target plane, and taking the determined projection part as a fragment part of the three-dimensional article model corresponding to the target designated article.

In an optional example, the respectively mapping the N three-dimensional article models by using the house panorama includes:

acquiring characteristic information of a target three-dimensional article model; wherein the target three-dimensional article model is any one of the N three-dimensional article models;

determining an image mapping mode matched with the target three-dimensional article model according to the characteristic information;

and according to the image mapping mode, mapping the target three-dimensional article model by using the house panoramic image.

In an optional example, the determining, according to the feature information, an image mapping manner matched with the target three-dimensional article model includes:

under the condition that the flatness of the designated surface of the target three-dimensional article model is determined to meet the preset condition according to the characteristic information, determining that an image mapping mode matched with the target three-dimensional article model is an automatic plane mapping mode;

or,

and under the condition that other three-dimensional article models do not exist in the initial three-dimensional house model within the preset distance range of the target three-dimensional article model according to the characteristic information, determining that an image mapping mode matched with the target three-dimensional article model is a virtual camera mapping mode.

In an optional example, the item identifying the house panorama to obtain an item identification result includes:

acquiring a pre-trained article identification model for identifying a specified article from an image;

and inputting the house panorama into the article identification model to obtain an article identification result output by the article identification model.

According to another aspect of the embodiments of the present disclosure, there is provided a three-dimensional house model building apparatus based on a house panorama, including:

the first generation module is used for generating an initial three-dimensional house model by utilizing the house panoramic image;

the first acquisition module is used for identifying articles of the house panoramic image to obtain an article identification result;

a second generating module, configured to generate, in the initial three-dimensional house model, N three-dimensional item models corresponding to N designated items in the house panorama according to the item identification result; wherein N is an integer greater than or equal to 1;

the processing module is used for respectively carrying out mapping processing on the N three-dimensional article models by utilizing the house panoramic image;

and the second acquisition module is used for acquiring a target three-dimensional house model according to the initial three-dimensional house model after the N three-dimensional article models are subjected to mapping processing.

In one optional example, the second generating module includes:

the first determining submodule is used for determining M reference points of a target specified article in the house panoramic image according to the article identification result; the target specified item is any one of N specified items in the house panoramic image, and M is an integer greater than or equal to 3;

a second determining submodule, configured to map the M reference points to the initial three-dimensional house model to determine M mapping points in the initial three-dimensional house model;

a third determining submodule, configured to determine a stretching reference plane in the initial three-dimensional house model according to the M mapping points;

the fourth determining submodule is used for determining the stretching parameters corresponding to the target specified article;

and the generation submodule is used for performing stretching processing on the stretching datum plane according to the stretching parameters corresponding to the target specified article so as to generate a three-dimensional article model corresponding to the target specified article in the initial three-dimensional house model.

In an optional example, the second obtaining module includes:

a fifth determining submodule, configured to determine fragment portions of the N three-dimensional article models subjected to mapping processing;

and the first obtaining submodule is used for deleting the determined fragment part from the initial three-dimensional house model after the N three-dimensional article models are subjected to mapping processing so as to obtain a target three-dimensional house model.

In one optional example, the stretch parameters include a stretch direction and a stretch distance:

the fifth determination submodule includes:

a first determination unit configured to determine a target plane; wherein the stretching direction is a direction pointing from the stretching reference plane to the target plane, and the target plane is parallel to the stretching reference plane and is at the stretching distance from the stretching reference plane;

a second determination unit configured to determine a surface of the three-dimensional article model corresponding to the target designated article, the surface being opposite to the stretching reference plane;

and the third determining unit is used for detecting the determined convex part of the surface relative to the target plane and taking the determined convex part as a fragment part of the three-dimensional article model corresponding to the target designated article.

In one optional example, the processing module includes:

the second obtaining submodule is used for obtaining the characteristic information of the target three-dimensional article model; wherein the target three-dimensional article model is any one of the N three-dimensional article models:

a sixth determining submodule, configured to determine, according to the feature information, an image mapping manner matched with the target three-dimensional article model:

and the processing submodule is used for carrying out mapping processing on the target three-dimensional article model by utilizing the house panoramic image according to the image mapping mode.

In an optional example, the sixth determining sub-module is specifically configured to:

under the condition that the flatness of the specified surface of the target three-dimensional article model is determined to meet the preset condition according to the characteristic information, determining that an image mapping mode matched with the target three-dimensional article model is an automatic plane mapping mode;

or,

In an optional example, the first obtaining module includes:

a third obtaining sub-module, configured to obtain a pre-trained item identification model for identifying a specified item from the image:

and the fourth obtaining submodule is used for inputting the house panorama into the article identification model so as to obtain an article identification result output by the article identification model.

According to still another aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium storing a computer program for executing the above-described three-dimensional house model building method based on a house panorama.

According to still another aspect of an embodiment of the present disclosure, there is provided an electronic device including:

a processor:

a memory for storing the processor-executable instructions:

and the processor is used for reading the executable instructions from the memory and executing the instructions to realize the building panorama-based three-dimensional building model construction method.

In the embodiment of the present disclosure, after the initial three-dimensional house model is generated by using the house panorama, N three-dimensional article models corresponding to N designated articles in the house panorama may be generated in the initial three-dimensional house model according to an article recognition result obtained by performing article recognition on the house panorama, in addition, the N three-dimensional article models may be respectively subjected to mapping by using the house panorama, and then, a target three-dimensional house model may be obtained according to the initial three-dimensional house model after mapping by using the N three-dimensional article models, and the target three-dimensional house model may be regarded as a three-dimensional house model finally constructed based on the house panorama. It should be noted that, the initial three-dimensional house model after the mapping processing of the N three-dimensional article models can not only reflect the spatial structure information of the real house, but also reflect the article information of the real house, so that the target three-dimensional house model obtained according to the initial three-dimensional house model after the mapping processing of the N three-dimensional article models can also simultaneously reflect the spatial structure information and the article information of the real house, therefore, compared with the related art, the three-dimensional house model constructed based on the house panorama in the embodiment of the disclosure can reflect richer information, and the display effect is more intuitive and real, so that the model display effect and the user experience can be effectively improved.

The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent by describing in more detail embodiments of the present disclosure with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure and not to limit the disclosure. In the drawings, like reference numbers generally represent like parts or steps.

Fig. 1 is a schematic flowchart of a method for building a three-dimensional house model based on a house panorama according to an exemplary embodiment of the present disclosure.

Fig. 2 is a schematic flowchart of a building model construction method based on a house panorama according to another exemplary embodiment of the present disclosure.

FIG. 3 is a partial schematic view of a target three-dimensional house model in an embodiment of the present disclosure.

Fig. 4 is a schematic flowchart of a building model construction method based on a house panorama according to still another exemplary embodiment of the present disclosure.

Fig. 5 is a flowchart illustrating a building model building method based on a house panorama according to another exemplary embodiment of the present disclosure.

Fig. 6 is a schematic structural diagram of a three-dimensional house model building device based on a house panorama according to an exemplary embodiment of the present disclosure.

Fig. 7 is a schematic structural diagram of a three-dimensional house model building device based on a house panorama according to another exemplary embodiment of the present disclosure.

Fig. 8 is a schematic structural diagram of a three-dimensional house model building device based on a house panorama according to still another exemplary embodiment of the present disclosure.

Fig. 9 is a block diagram of an electronic device provided in an exemplary embodiment of the present disclosure.

Detailed Description

Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are only some of the embodiments of the present disclosure, and not all of the embodiments of the present disclosure, and it is to be understood that the present disclosure is not limited by the example embodiments described herein.

It should be noted that: the relative arrangement of parts and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present disclosure are used merely to distinguish one element from another, and are not intended to imply any particular technical meaning, nor is the necessary logical order between them.

It is also understood that in embodiments of the present disclosure, "a plurality" may refer to two or more and "at least one" may refer to one, two or more.

It is also to be understood that any reference to any component, data, or structure in the embodiments of the disclosure, may be generally understood as one or more, unless explicitly defined otherwise or stated otherwise.

In addition, the term "and/or" in the present disclosure is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the former and latter associated objects are in an "or" relationship.

It should also be understood that the description of the various embodiments of the present disclosure emphasizes the differences between the various embodiments, and the same or similar parts may be referred to each other, so that the descriptions thereof are omitted for brevity.

Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.

Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

The disclosed embodiments may be applied to electronic devices such as terminal devices, computer systems, servers, etc., which are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known terminal devices, computing systems, environments, and/or configurations that may be suitable for use with electronic devices, such as terminal devices, computer systems, servers, and the like, include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set top boxes, programmable consumer electronics, network pcs, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above systems, and the like.

Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

Exemplary method

Fig. 1 is a schematic flowchart of a method for building a three-dimensional house model based on a house panorama according to an exemplary embodiment of the present disclosure. The method shown in fig. 1 includes step 101, step 102, step 103, step 104 and step 105, and each step is described below.

And 101, generating an initial three-dimensional house model by using the house panoramic image.

Here, the real house may be photographed from a plurality of different angles by using an image photographing apparatus such as a camera to obtain a plurality of photographed images, for example, the real house may be photographed from 6 different angles to obtain 6 photographed images.

Next, these 6 shot images may be subjected to image stitching or the like to generate 1 house panorama, and the generated house panorama may be used for generation of the initial three-dimensional house model. It should be noted that the initial three-dimensional house model can only represent the spatial structure information of the real house, but cannot represent other aspects of the real house, and the initial three-dimensional house model can also be regarded as the spatial structure model of the real house.

And 102, identifying the articles of the house panoramic image to obtain an article identification result.

Here, the article identification technology may be adopted to perform article identification on the house panorama to obtain an article identification result. Alternatively, with the article identification technology, only the specified article in the panoramic image of the house can be identified, and the specified article can be some large articles, such as a bed, a sofa, a tea table, a dining table, and the like.

103, generating N three-dimensional item models corresponding to N specified items in the house panoramic image in the initial three-dimensional house model according to the item identification result; wherein N is an integer greater than or equal to 1.

Here, N may be 1, 3, 5, 8, 10, 15, etc., and is not listed here.

After the item identification result is obtained, all the designated items in the house panorama can be determined according to the item identification result, and on this basis, the N designated items in step 103 can be further determined, specifically, the N designated items can include all the designated items in the house panorama, or the N designated items can include only part of the designated items in the house panorama.

Next, for each of the N designated items, a corresponding three-dimensional item model may be generated at a suitable location in the initial three-dimensional house model, so that the initial three-dimensional house model may have N three-dimensional item models, and there may be a one-to-one correspondence between the N designated items and the N three-dimensional item models.

And step 104, respectively carrying out mapping processing on the N three-dimensional article models by using the house panoramic image.

Here, the N three-dimensional article models may be mapped by using the house panorama by an image mapping method including, but not limited to, an automatic texture mapping method, an automatic plane mapping method, a virtual camera mapping method, and the like.

It can be understood that the implementation principle of the automatic texture mapping method may be: firstly, on a voxel three-dimensional reconstruction model, establishing a Gaussian mixture model for each target surface voxel to describe the distribution of the observed values of the color of the voxel on different images; then, by using the input sequence images and corresponding camera parameters and combining with the visibility constraint of a scene, updating the Gaussian mixture model of the corresponding voxel in the three-dimensional scene model by using the color information of each pixel of the images; and finally, estimating the actual color texture of the voxel by using the expectation of the Gaussian mixture model to obtain an accurate and fine three-dimensional building texture model.

The automatic plane mapping mode can be realized according to the following principle: firstly, deleting uv coordinates (which is short for uv texture map coordinates and defines information of the position of each point on an image) from an input structure; then calculating new uv coordinates in the vertex shader, specifically, the uv coordinates may be set to x and z values of the object coordinates; then, performing adjustable tiling processing to repair the scaling and translation of the texture; texture coordinates based on world space may then be determined, and in particular, the location of the vertex coordinates in world space may be obtained and used to set the uv coordinates.

The implementation principle of the virtual camera mapping mode can be as follows: calibrating and setting internal parameters and external parameters of a physical camera and a virtual camera; respectively calculating rotation matrixes converted from a world coordinate system to a virtual camera coordinate system and a physical camera coordinate system; establishing a relation between the virtual image coordinate and the world coordinate; calculating the intersection point of the connecting line of the virtual camera coordinate origin and a certain point of the world coordinate and the projection plane; converting the intersection point from a world coordinate system to a physical image coordinate system; establishing a relation between the virtual image coordinates and the physical image coordinates; and establishing a mapping relation for each point in the virtual image to obtain a mapping table Map, and looking up the table to obtain the virtual image.

And 105, obtaining a target three-dimensional house model according to the initial three-dimensional house model after the mapping processing of the N three-dimensional article models.

Here, after the N three-dimensional commodity models are respectively subjected to mapping processing based on the house panorama, an initial three-dimensional house model of the N three-dimensional commodity models subjected to mapping processing may be directly used as a target three-dimensional house model; or, further model optimization processing can be performed on the initial three-dimensional house model after the mapping processing is performed on the N three-dimensional article models, so as to obtain the target three-dimensional house model.

On the basis of the embodiment shown in fig. 1, as shown in fig. 2, step 103 includes:

step 1031, determining M reference points of the target specified item in the house panoramic image according to the item identification result; the target specified item is any one of N specified items in the house panorama, and M is an integer greater than or equal to 3.

Here, N may be 3, 4, 5, 6, etc., and is not listed here.

It should be noted that the item identification result may include location information of the target specified item in the house panoramic image. Assuming that the target specifies that the object is a bed, the bed in the house panorama can be simulated as a rectangle, and the object identification result can specifically include the position coordinates of 4 corner points of the rectangle, where the 4 corner points can be used as the M reference points in step 1031.

Step 1032, map the M fiducial points to the initial three-dimensional house model to determine M mapped points in the initial three-dimensional house model.

Since the initial three-dimensional house model is generated using the house panorama, a conversion relationship (e.g., a conversion matrix) between the house panorama and the initial three-dimensional house model can be determined through analysis and calculation. Thus, after the M reference points in step 1031 are determined, the M reference points may be mapped one by one to the initial three-dimensional house model by using the transformation relation to determine M mapping points in the initial three-dimensional house model, which correspond one by one to the M reference points.

And step 1033, determining a stretching datum plane in the initial three-dimensional house model according to the M mapping points.

Here, after the M mapped points in the initial three-dimensional house model are determined, a plane having the M mapped points as corner points may be determined and used as a stretching reference plane in the initial three-dimensional house model.

In a specific example, the object specifies that the object is a bed, the object corresponds to the bed, the object identification result specifically includes position coordinates of 4 corner points, the 4 corner points are a point a, a point B, a point C, and a point D, respectively, by performing step 1032, determining that the mapping point corresponding to the point a is a point a ', the mapping point corresponding to the point B is a point B', the mapping point corresponding to the point C is a point C ', and the mapping point corresponding to the point D is a point D', a plane a 'B' C 'D' may be determined in the initial three-dimensional house model, and the plane a 'B' C 'D' may be used as the stretching reference plane.

Step 1034, determining the stretching parameters corresponding to the target designated article.

Here, the stretching reference includes, but is not limited to, stretching direction, stretching distance, and the like.

Alternatively, the stretching direction corresponding to the target designated item may be determined according to the placement condition of the target designated item in the real house, for example, the bed is generally horizontally placed on the floor, then, in the case that the target designated object is a bed, the stretching direction corresponding to the target designated item may be a vertically upward direction, and in the case that the target designated item is another item, the stretching direction corresponding to the target designated item may be a vertically upward direction or another direction.

Alternatively, a stretching distance may be preset, and the stretching distance may be used as a stretching distance corresponding to each designated article, and of course, different stretching distances may also be preset for different designated articles according to actual situations.

And 1035, stretching the stretching datum plane according to the stretching parameters corresponding to the target specified object so as to generate a three-dimensional object model corresponding to the target specified object in the initial three-dimensional house model.

Here, by performing the stretching process on the stretching reference plane in accordance with the stretching parameter corresponding to the target designated article, it is possible to stretch a three-dimensional article model having a stereoscopic effect, and the stretched three-dimensional article model is the three-dimensional article model corresponding to the target designated article. In the case where the target-designated item is a bed, since the bed in the house panorama is simulated as a rectangle, the three-dimensional item model corresponding to the target-designated item may be specifically a rectangular parallelepiped model.

Therefore, in the embodiment of the disclosure, for a target specified article in a house panorama, based on the point mapping processing from the house panorama to the initial three-dimensional house model, a suitable stretching datum plane can be determined in the initial three-dimensional house model, and then the stretching datum plane is combined to a suitable stretching parameter determined for the target specified article, so that a three-dimensional article model corresponding to the target specified article can be generated in the initial three-dimensional house model conveniently and reliably through the stretching processing.

In an alternative example, obtaining a target three-dimensional house model from the initial three-dimensional house model after the mapping process of the N three-dimensional article models comprises:

Here, after the mapping process is performed on each of the N three-dimensional commodity models from the house panorama, the fragment portion of each of the N three-dimensional commodity models subjected to the mapping process may be specified, and the fragment portion of any one three-dimensional commodity model may be regarded as a portion that should not be present on the own three-dimensional commodity model.

In one embodiment, the stretch parameters include the stretch direction and the stretch distance;

determining respective fragment parts of the N three-dimensional article models subjected to mapping processing, wherein the fragment parts comprise:

determining a target plane; the stretching direction is the direction from the stretching reference surface to the target plane, the target plane is parallel to the stretching reference surface, and the distance between the target plane and the stretching reference surface is the stretching distance;

determining the surface, opposite to the stretching datum plane, of the three-dimensional article model corresponding to the target specified article;

and detecting a convex part of the determined surface relative to the target plane, and taking the determined convex part as a fragment part of the three-dimensional article model corresponding to the target designated article.

It should be noted that, when the stretching reference plane is subjected to the stretching process according to the stretching parameters including the stretching direction and the stretching distance to generate the three-dimensional article model corresponding to the target specified article, ideally, the surface of the three-dimensional article model corresponding to the target specified article, which is opposite to the stretching reference plane, should be completely parallel to the stretching reference plane, and then the target plane, which is parallel to the stretching reference plane and is at the stretching distance from the stretching reference plane, should be completely flush with the surface. In view of this, in this embodiment, the protruding portion of the surface relative to the target plane may be determined, the determined protruding portion may be regarded as a portion that should not exist on the three-dimensional article model corresponding to the target specified article, and the determined protruding portion may be regarded as a fragment portion of the three-dimensional article model corresponding to the target specified article.

In a similar manner as in the above paragraph, fragment portions of other three-dimensional commodity models in the N three-dimensional commodity models may also be determined, and then, all the determined fragment portions may be deleted from the initial three-dimensional house model after the mapping process of the N three-dimensional commodity models to obtain the target three-dimensional house model, and a partial schematic diagram of the target three-dimensional house model may be as shown in fig. 3.

Therefore, in the embodiment of the disclosure, the accuracy and the reliability of the finally obtained target three-dimensional house model can be better ensured through the determining operation and the deleting operation of the fragment part, so that the model display effect and the user experience are further promoted.

Based on the embodiment shown in fig. 1, as shown in fig. 4, step 104 includes:

step 1041, acquiring feature information of the target three-dimensional article model; the target three-dimensional article model is any one of the N three-dimensional article models.

Here, the characteristic information of the target three-dimensional article model includes, but is not limited to, size information of the target three-dimensional article model, flatness of each surface of the target three-dimensional article model, boundary position information of the target three-dimensional article model, and the like; the boundary position information can be characterized by boundary coordinates.

And 1042, determining an image mapping mode matched with the target three-dimensional object model according to the characteristic information.

It should be noted that, the specific implementation of step 1042 is various, and is described below by way of example.

In one embodiment, step 1042, comprises:

here, the target specified item may be a bed, and the target three-dimensional item model may be a three-dimensional item model corresponding to the bed, and in this case, the specified surface of the target three-dimensional item model may be a surface of the three-dimensional item model corresponding to the target specified item mentioned above, which is opposite to the stretching reference surface.

In particular, the flatness of the surface opposite to the stretching reference plane may be extracted from the feature information, and the flatness may be compared with a preset flatness. If the flatness is larger than the preset flatness, the flatness can be considered to meet the preset condition, and the automatic plane mapping mode is applicable to the target three-dimensional article model, so that the image mapping mode matched with the target three-dimensional article model can be determined to be the automatic plane mapping mode. If the flatness is less than or equal to the preset flatness, the flatness is considered not to meet the preset condition, and the automatic plane mapping mode is not applicable to the target three-dimensional article model, so that the image mapping mode matched with the target three-dimensional article model can be determined to be other image mapping modes, such as an automatic texture mapping mode.

In another embodiment, step 1042 includes:

and under the condition that other three-dimensional article models do not exist in the preset distance range of the target three-dimensional article model in the initial three-dimensional house model according to the characteristic information, determining that the image mapping mode matched with the target three-dimensional article model is a virtual camera mapping mode.

In specific implementation, boundary position information of the target three-dimensional article model can be extracted from the characteristic information, and then whether other three-dimensional article models exist in the initial three-dimensional house model within a preset distance range of the target three-dimensional article model or not can be identified by combining the boundary position information. If no other three-dimensional article model exists in the preset distance range of the target three-dimensional article model, which indicates that the boundary between the target three-dimensional article model and the other three-dimensional article model is obvious enough, the virtual camera mapping mode is applicable to the target three-dimensional article model, so that the image mapping mode matched with the target three-dimensional article model can be determined to be the virtual camera mapping mode. If other three-dimensional article models exist in the preset distance range of the target three-dimensional article model, the boundary between the target three-dimensional article model and the other three-dimensional article models is possibly not obvious enough, so that the image mapping mode matched with the target three-dimensional article model can be determined to be other image mapping modes, such as an automatic texture mapping mode.

In the above two embodiments, an appropriate image mapping manner may be selected for the target three-dimensional article model according to the feature information of the target three-dimensional article model.

And 1043, according to the image mapping mode, performing mapping processing on the target three-dimensional article model by using the house panoramic image.

Here, after determining the image mapping manner matched with the target three-dimensional article model, the panorama can be performed on the target three-dimensional house model according to the determined image mapping manner, so that the target three-dimensional article model can have a real texture effect, and the target three-dimensional article model can look more real.

Therefore, in the embodiment of the disclosure, the image mapping mode is determined according to the characteristic information of the target three-dimensional article model, and the mapping processing of the target three-dimensional article model is performed according to the determined image mapping mode, so that the mapping effect can be effectively ensured, and the model display effect and the user experience can be further improved.

Based on the embodiment shown in fig. 1, as shown in fig. 5, step 102 includes:

step 1021, an item identification model trained in advance and used for identifying the specified item from the image is obtained.

Here, each specific object to be identified may be determined, for example, a bed, a sofa, a tea table, or a dining table as the specific object to be identified. Next, panoramic views for model training may be acquired, and beds, sofas, tea tables, and tables appearing in these panoramic views may be labeled manually, after which model training may be performed through machine learning using the labeled panoramic views to obtain an article recognition model for recognizing a specified article from an image.

And step 1022, inputting the house panoramic image into the item identification model to obtain an item identification result output by the item identification model.

Here, the house panorama may be provided as input data to the article recognition model, and in this case, the article recognition model may automatically recognize the designated article and output a corresponding article recognition result, which may include location information, type information, and the like of each of the recognized designated articles.

It can be seen that, in the embodiments of the present disclosure, by using the article recognition model to perform article recognition, the efficiency of obtaining the article recognition result can be effectively ensured, and since the article recognition model is trained using very objective data, the accuracy and reliability of the article recognition result can be effectively ensured.

In summary, in the embodiment of the present disclosure, based on the house panorama, the specified articles may be identified by using an article identification technology, a three-dimensional article model corresponding to each identified specified article may be generated through virtual modeling, and the three-dimensional article model obtained through virtual modeling may be subjected to panorama mapping processing, so that a three-dimensional house model (i.e., the above target three-dimensional house model) with a very intuitive and real display effect may be obtained, and a user may view a more refined indoor model effect at the C-terminal.

Any of the house panorama-based three-dimensional house model construction methods provided by embodiments of the present disclosure may be performed by any suitable device having data processing capabilities, including but not limited to: terminal equipment, a server and the like. Alternatively, any one of the house panorama-based three-dimensional house model building methods provided by the embodiments of the present disclosure may be executed by a processor, for example, the processor may execute any one of the house panorama-based three-dimensional house model building methods mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. Which will not be described in detail below.

Exemplary devices

Fig. 6 is a schematic structural diagram of an apparatus for building a three-dimensional house model based on a house panorama according to an exemplary embodiment of the present disclosure, and the apparatus shown in fig. 6 includes a first generating module 601, a first obtaining module 602, a second generating module 603, a processing module 604, and a second obtaining module 605.

A first generating module 601, configured to generate an initial three-dimensional house model by using a house panorama;

a first obtaining module 602, configured to perform item identification on the house panorama to obtain an item identification result;

a second generating module 603, configured to generate, in the initial three-dimensional house model, N three-dimensional item models corresponding to N designated items in the house panorama according to the item identification result; wherein N is an integer greater than or equal to 1;

the processing module 604 is configured to perform mapping processing on the N three-dimensional article models respectively by using the house panorama;

a second obtaining module 605, configured to obtain a target three-dimensional house model according to the initial three-dimensional house model after the mapping processing is performed on the N three-dimensional article models.

In an alternative example, as shown in fig. 7, the second generating module 603 includes:

a first determining sub-module 6031, configured to determine, according to the item identification result, M reference points of the target specified item in the house panorama; the target specified article is any one of N specified articles in the house panoramic image, and M is an integer greater than or equal to 3;

a second determining submodule 6032 for mapping the M reference points to the initial three-dimensional house model to determine M mapping points in the initial three-dimensional house model;

a third determining submodule 6033, configured to determine, according to the M mapping points, a stretching reference plane in the initial three-dimensional house model;

a fourth determining submodule 6034, configured to determine a stretching parameter corresponding to the target specified item;

the generating sub-module 6035 is configured to perform stretching processing on the stretching reference plane according to the stretching parameter corresponding to the target specified item, so as to generate a three-dimensional item model corresponding to the target specified item in the initial three-dimensional house model.

In an optional example, the second obtaining module 605 includes:

the fifth determining submodule is used for determining fragment parts of the N three-dimensional article models subjected to mapping processing;

and the first obtaining submodule is used for deleting the determined fragment part from the initial three-dimensional house model after the mapping processing is carried out on the N three-dimensional article models so as to obtain the target three-dimensional house model.

In an alternative example, the stretch parameters include a stretch direction and a stretch distance;

a fifth determination submodule comprising:

a first determination unit configured to determine a target plane; the stretching direction is the direction from the stretching reference plane to the target plane, the target plane is parallel to the stretching reference plane, and the distance between the target plane and the stretching reference plane is the stretching distance;

In an alternative example, as shown in fig. 8, the processing module 604 includes:

a second obtaining submodule 6041, configured to obtain feature information of the target three-dimensional article model; the target three-dimensional article model is any one of the N three-dimensional article models;

a sixth determining submodule 6042, configured to determine, according to the feature information, an image mapping manner that matches the target three-dimensional article model;

and the processing submodule 603 is configured to perform mapping processing on the target three-dimensional article model by using the house panorama according to an image mapping manner.

In an alternative example, the sixth determining sub-module 6042 is specifically configured to:

or,

In an optional example, the first obtaining module 602 includes:

the third acquisition sub-module is used for acquiring a pre-trained article identification model used for identifying a specified article from the image;

and the fourth acquisition submodule is used for inputting the house panorama into the article identification model so as to obtain an article identification result output by the article identification model.

Exemplary electronic device

Next, an electronic apparatus according to an embodiment of the present disclosure is described with reference to fig. 9. The electronic device may be either or both of the first device and the second device, or a stand-alone device separate from them, which stand-alone device may communicate with the first device and the second device to receive the acquired input signals therefrom.

Fig. 9 illustrates a block diagram of an electronic device 900 in accordance with an embodiment of the disclosure.

As shown in fig. 9, the electronic device 900 includes one or more processors 901 and memory 902.

The processor 901 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 900 to perform desired functions.

Memory 902 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by the processor 901 to implement the house panorama-based three-dimensional house model building method of the various embodiments of the present disclosure described above and/or other desired functions. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.

In one example, the electronic device 900 may further include: an input device 903 and an output device 904, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

For example, when the electronic device 900 is a first device or a second device, the input apparatus 903 may be a microphone or a microphone array. When the electronic device 900 is a stand-alone device, the input means 903 may be a communication network connector for receiving the acquired input signals from the first device and the second device.

The input device 903 may also include, for example, a keyboard, a mouse, and the like.

The output device 904 can output various information to the outside. The output devices 904 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, among others.

Of course, for simplicity, only some of the components of the electronic device 900 relevant to the present disclosure are shown in fig. 9, omitting components such as buses, input/output interfaces, and the like. In addition, electronic device 900 may include any other suitable components depending on the particular application.

Exemplary computer program product and computer-readable storage Medium

In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in a method of building a three-dimensional house model based on a house panorama according to various embodiments of the present disclosure described in the "exemplary methods" section of this specification above.

The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps in the house panorama based three-dimensional house model building method according to various embodiments of the present disclosure described in the "exemplary methods" section above in this specification.

The computer readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure will be described in detail with reference to specific details.

In the present specification, the embodiments are described in a progressive manner, and each embodiment focuses on differences from other embodiments, and the same or similar parts in each embodiment are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and reference may be made to the partial description of the method embodiment for relevant points.

The block diagrams of devices, apparatuses, systems referred to in this disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably herein. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".

The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

It is also noted that in the devices, apparatuses, and methods of the present disclosure, each component or step can be decomposed and/or recombined. Such decomposition and/or recombination should be considered as equivalents of the present disclosure.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, the description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims

1. A building method of a three-dimensional house model based on a house panorama is characterized by comprising the following steps:

generating an initial three-dimensional house model by using the house panorama, wherein the initial three-dimensional house model is a space structure model only reflecting space structure information of a real house;

according to the item identification result, generating N three-dimensional item models corresponding to N specified items in the house panoramic image in the initial three-dimensional house model; wherein N is an integer greater than or equal to 1;

respectively carrying out mapping processing on the N three-dimensional article models by using the house panoramic image; and

obtaining a target three-dimensional house model according to the initial three-dimensional house model after the N three-dimensional article models are subjected to mapping processing, wherein the target three-dimensional house model embodies the space structure information and the article information of a real house;

generating N three-dimensional item models corresponding to N designated items in the house panorama in the initial three-dimensional house model according to the item identification result, wherein the method comprises the following steps:

determining M reference points of a target designated item in the house panoramic image according to the item identification result; the target specified article is any one of N specified articles in the house panoramic image, and M is an integer greater than or equal to 3;

determining a stretching parameter corresponding to the target specified article; the stretching parameters comprise a stretching direction and a stretching distance, the stretching direction is determined according to the placing condition of the target designated article in the real house, and the stretching distance is a distance preset for the target designated article;

2. The method according to claim 1, wherein said obtaining a target three-dimensional house model from said initial three-dimensional house model after said mapping process of said N three-dimensional commodity models comprises:

3. The method of claim 2, wherein the determining the patch portions of the mapped N three-dimensional object models comprises:

and detecting a convex part of the determined surface relative to the target plane, and taking the determined convex part as a fragment part of a three-dimensional article model corresponding to the target designated article.

4. The method according to claim 1, wherein said using the house panorama to map the N three-dimensional item models respectively comprises:

5. The method according to claim 4, wherein the determining an image mapping manner matched with the target three-dimensional article model according to the feature information comprises:

or,

6. The method according to any one of claims 1 to 5, wherein the item identification of the house panorama to obtain an item identification result comprises:

and inputting the house panorama into the item identification model to obtain an item identification result output by the item identification model.

7. A three-dimensional house model building device based on house panorama, characterized by that includes:

the system comprises a first generation module, a second generation module and a third generation module, wherein the first generation module is used for generating an initial three-dimensional house model by utilizing a house panorama, and the initial three-dimensional house model is a spatial structure model only reflecting the spatial structure information of a real house;

a second generating module, configured to generate, in the initial three-dimensional house model according to the item identification result, N three-dimensional item models corresponding to N designated items in the house panorama; wherein N is an integer greater than or equal to 1;

the processing module is used for respectively carrying out mapping processing on the N three-dimensional article models by utilizing the house panoramic image; and

the second obtaining module is used for obtaining a target three-dimensional house model according to the initial three-dimensional house model after the N three-dimensional article models are subjected to mapping processing, wherein the target three-dimensional house model reflects the space structure information and the article information of a real house;

wherein the second generating module comprises:

a second determining sub-module for mapping the M reference points to the initial three-dimensional house model to determine M mapping points in the initial three-dimensional house model;

the fourth determining submodule is used for determining the stretching parameters corresponding to the target specified article; the stretching parameters comprise a stretching direction and a stretching distance, the stretching direction is determined according to the placing condition of the target designated article in the real house, and the stretching distance is a distance preset for the target designated article;

8. The apparatus of claim 7, wherein the second obtaining module comprises:

9. The apparatus of claim 8, wherein the fifth determining submodule comprises:

a second determination unit configured to determine a surface of the three-dimensional article model corresponding to the target specified article, the surface being opposite to the stretching reference plane;

10. The apparatus of claim 7, wherein the processing module comprises:

the second acquisition sub-module is used for acquiring the characteristic information of the target three-dimensional article model; wherein the target three-dimensional article model is any one of the N three-dimensional article models;

a sixth determining submodule, configured to determine, according to the feature information, an image mapping manner matched with the target three-dimensional article model;

11. The apparatus according to claim 10, wherein the sixth determining submodule is specifically configured to:

or,

12. The apparatus according to any one of claims 7 to 11, wherein the first obtaining module comprises:

a third obtaining sub-module, configured to obtain a pre-trained article recognition model for recognizing a specific article from the image;

and the fourth acquisition sub-module is used for inputting the house panoramic image into the article identification model so as to obtain an article identification result output by the article identification model.

13. A computer-readable storage medium storing a computer program for executing the method for building a three-dimensional house model based on a house panorama of any one of claims 1-6.

14. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

the processor is used for reading the executable instructions from the memory and executing the instructions to realize the three-dimensional house model building method based on the house panorama of any one of the claims 1-6.