CN112233103B

CN112233103B - Three-dimensional house model quality evaluation method and device and computer readable storage medium

Info

Publication number: CN112233103B
Application number: CN202011159447.5A
Authority: CN
Inventors: 付文兰; 董秋成; 戴良斌
Original assignee: Seashell Housing Beijing Technology Co Ltd
Current assignee: Seashell Housing Beijing Technology Co Ltd
Priority date: 2020-10-26
Filing date: 2020-10-26
Publication date: 2021-09-21
Anticipated expiration: 2040-10-26
Also published as: CN112233103A

Abstract

The embodiment of the disclosure discloses a method and a device for evaluating the quality of a three-dimensional house model and a computer-readable storage medium. The method comprises the following steps: obtaining N two-dimensional house images corresponding to N visual angles according to the three-dimensional house model; wherein N is an integer greater than or equal to 2; inputting the N two-dimensional house images into a neural network to obtain N model quality evaluation data which are output by the neural network and correspond to the N two-dimensional house images; and outputting a model quality evaluation result of the three-dimensional house model according to the N visual angles and the N model quality evaluation data. Compared with the related art, the application range of the embodiment of the disclosure is wider, and the model quality evaluation result obtained by the embodiment of the disclosure and the model quality evaluation result of artificial subjectivity can have higher consistency, so that the actual requirement can be better met.

Description

Three-dimensional house model quality evaluation method and device and computer readable storage medium

Technical Field

The present disclosure relates to the field of three-dimensional modeling technologies, and in particular, to a method and an apparatus for evaluating quality of a three-dimensional house model, and a computer-readable storage medium.

Background

The application of the three-dimensional house model is more and more common, in some cases, the model quality evaluation of the three-dimensional house model is required, and currently, the common evaluation methods are as follows: in the modeling process, the characteristic value of the model is calculated, meanwhile, a scoring evaluation standard is formulated according to the big data statistical result, and evaluation is carried out based on the scoring evaluation standard and the characteristic value, for example, the size of the shielded part of the model after modeling is found by comparing the point cloud directly collected with the point cloud regenerated based on 3D mesh (namely, three-dimensional network), and the shielded part is compared with the empirical value, so that the shielding condition of the model is obtained.

It should be noted that different three-dimensional house models may be obtained based on different acquisition devices or different reconstruction algorithms, and the above evaluation method cannot unify the computation logics of different devices or different algorithms, for example, it may only be applied to the model quality evaluation of a three-dimensional house model obtained based on a certain acquisition device, and therefore, the above evaluation method has a strong limitation.

Disclosure of Invention

The present disclosure is proposed to solve the above technical problems. The embodiment of the disclosure provides a method and a device for evaluating the quality of a three-dimensional house model and a computer-readable storage medium.

According to an aspect of the embodiments of the present disclosure, there is provided a method for evaluating quality of a three-dimensional house model, including:

obtaining N two-dimensional house images corresponding to N visual angles according to the three-dimensional house model; wherein N is an integer greater than or equal to 2;

inputting the N two-dimensional house images into a neural network to obtain N model quality evaluation data which are output by the neural network and correspond to the N two-dimensional house images;

and outputting a model quality evaluation result of the three-dimensional house model according to the N visual angles and the N model quality evaluation data.

In an optional example, the outputting a model quality evaluation result of the three-dimensional house model according to the N view angles and the N model quality evaluation data includes:

obtaining M probability groups corresponding to M model quality evaluation dimensions for each two-dimensional house image in the N two-dimensional house images according to the N model quality evaluation data; wherein, the probability group corresponding to any model quality evaluation dimension comprises: the probability of each corresponding evaluation grade under the quality evaluation dimension of the model is M, which is an integer greater than or equal to 1;

determining M evaluation scores corresponding to the M model quality evaluation dimensions for each two-dimensional house image according to the M probability groups obtained for each two-dimensional house image;

and outputting a model quality evaluation result of the three-dimensional house model according to the N visual angles and the M evaluation scores determined for each two-dimensional house image.

In an optional example, the determining, for each two-dimensional house image, M evaluation scores corresponding to the M model quality evaluation dimensions according to M probability groups obtained for each two-dimensional house image includes:

obtaining weights corresponding to all evaluation grades under the quality evaluation dimension of the target model; wherein the target model quality assessment dimension is any one of the M model quality assessment dimensions;

according to the obtained weight, carrying out weighted summation on all probabilities in M probability groups determined for the target two-dimensional house image and in a probability group corresponding to the target model quality evaluation dimension to obtain a weighted summation result; the target two-dimensional house image is any one of the N two-dimensional house images;

and taking the weighted summation result as an evaluation score corresponding to the target model quality evaluation dimension determined for the target two-dimensional house image.

In an optional example, the model quality assessment result comprises: and each view angle in the N view angles and the mapping relation between the M evaluation scores determined for the two-dimensional house image corresponding to each view angle in the N two-dimensional house images.

In an optional example, the obtaining N two-dimensional house images corresponding to N views according to the three-dimensional house model includes:

displaying the three-dimensional house model with a black background;

and performing screenshot processing on the three-dimensional house model displayed by the black background from N visual angles to obtain N two-dimensional house images corresponding to the N visual angles.

In an alternative example, the inputting the N two-dimensional house images into a neural network includes:

removing the edge invalid black area in each two-dimensional house image in the N two-dimensional house images;

and inputting the N two-dimensional house images with the edge invalid black areas removed into a neural network.

In an alternative example, the neural network is trained by using a sample two-dimensional house image obtained from a sample three-dimensional house model and model quality labeling data of the sample two-dimensional house image.

According to another aspect of the embodiments of the present disclosure, there is provided a three-dimensional house model quality evaluation apparatus including:

the first acquisition module is used for acquiring N two-dimensional house images corresponding to N visual angles according to the three-dimensional house model; wherein N is an integer greater than or equal to 2;

the second acquisition module is used for inputting the N two-dimensional house images into a neural network so as to acquire N pieces of model quality evaluation data which are output by the neural network and correspond to the N two-dimensional house images;

and the output module is used for outputting the model quality evaluation result of the three-dimensional house model according to the N visual angles and the N model quality evaluation data.

In one optional example, the output module includes:

the first obtaining submodule is used for obtaining M probability groups corresponding to M model quality evaluation dimensions for each two-dimensional house image in the N two-dimensional house images according to the N model quality evaluation data; wherein, the probability group corresponding to any model quality evaluation dimension comprises: the probability of each corresponding evaluation grade under the quality evaluation dimension of the model is M, which is an integer greater than or equal to 1;

the determining submodule is used for determining M evaluation scores corresponding to the M model quality evaluation dimensions for each two-dimensional house image according to the M probability groups obtained for each two-dimensional house image;

and the output sub-module is used for outputting a model quality evaluation result of the three-dimensional house model according to the N visual angles and the M evaluation scores determined for each two-dimensional house image.

In one optional example, the determining sub-module includes:

the first acquisition unit is used for acquiring weights corresponding to all evaluation grades under the quality evaluation dimensionality of the target model; wherein the target model quality assessment dimension is any one of the M model quality assessment dimensions;

the second acquisition unit is used for carrying out weighted summation on all probabilities in M probability groups determined for the target two-dimensional house image and in the probability group corresponding to the target model quality evaluation dimension according to the obtained weights so as to obtain a weighted summation result; the target two-dimensional house image is any one of the N two-dimensional house images;

and the determining unit is used for taking the weighted summation result as an evaluation score which is determined for the target two-dimensional house image and corresponds to the target model quality evaluation dimension.

In an optional example, the first obtaining module includes:

the display submodule is used for displaying the three-dimensional house model by using a black background;

and the second acquisition submodule is used for carrying out screenshot processing on the three-dimensional house model displayed by the black background from the N visual angles so as to obtain N two-dimensional house images corresponding to the N visual angles.

In an optional example, the second obtaining module includes:

a eliminating unit for eliminating the edge invalid black area in each two-dimensional house image in the N two-dimensional house images;

and the output unit is used for inputting the N two-dimensional house images with the edge invalid black areas removed into a neural network.

According to still another aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium storing a computer program for executing the above-described three-dimensional house model quality evaluation method.

According to still another aspect of an embodiment of the present disclosure, there is provided an electronic device including:

a processor;

a memory for storing the processor-executable instructions;

and the processor is used for reading the executable instructions from the memory and executing the instructions to realize the three-dimensional house model quality evaluation method.

In the embodiment of the disclosure, N two-dimensional house images corresponding to N viewing angles may be obtained according to the three-dimensional house model, then the N two-dimensional house images are input to the neural network to obtain N model quality assessment data output by the neural network and corresponding to the N two-dimensional house images, and then a model quality assessment result of the three-dimensional house model may be output according to the N viewing angles and the N model quality assessment data, so that model quality assessment of the three-dimensional house model is achieved. It should be noted that no matter what kind of acquisition equipment and what kind of reconstruction algorithm are used to obtain the three-dimensional house model, the two-dimensional house image can be obtained by adopting a uniform obtaining mode, and the obtained three-dimensional house image is uniformly processed through the neural network, so as to realize model quality evaluation, that is, the model quality evaluation method in the embodiment of the present disclosure can be applied to three-dimensional house models obtained based on various acquisition devices and various reconstruction algorithms, and therefore, compared with the related art, the application range of the embodiment of the present disclosure is wider, moreover, the visual angle corresponding to the two-dimensional house image can be a visual angle commonly used by a user, so that the model quality evaluation result obtained by the embodiment of the disclosure and the model quality evaluation result of artificial subjectivity can have higher consistency, and the actual requirement can be better met.

The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent by describing in more detail embodiments of the present disclosure with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure and not to limit the disclosure. In the drawings, like reference numbers generally represent like parts or steps.

Fig. 1 is a schematic flow chart of a three-dimensional house model quality evaluation method according to an exemplary embodiment of the present disclosure.

Fig. 2 is a schematic flow chart of a three-dimensional house model quality evaluation method according to another exemplary embodiment of the present disclosure.

Fig. 3 is a schematic flow chart of a three-dimensional house model quality evaluation method according to still another exemplary embodiment of the present disclosure.

FIG. 4 is a schematic illustration of a two-dimensional house image in an exemplary embodiment of the disclosure.

Fig. 5 is a schematic diagram of the operation of a neural network in an exemplary embodiment of the present disclosure.

Fig. 6 is a schematic flow chart of a three-dimensional house model quality evaluation method according to still another exemplary embodiment of the present disclosure.

Fig. 7 is a schematic structural diagram of a three-dimensional house model quality evaluation device according to an exemplary embodiment of the present disclosure.

Fig. 8 is a block diagram of an electronic device provided in an exemplary embodiment of the present disclosure.

Detailed Description

Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of the embodiments of the present disclosure and not all embodiments of the present disclosure, with the understanding that the present disclosure is not limited to the example embodiments described herein.

It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present disclosure are used merely to distinguish one element from another, and are not intended to imply any particular technical meaning, nor is the necessary logical order between them.

It is also understood that in embodiments of the present disclosure, "a plurality" may refer to two or more and "at least one" may refer to one, two or more.

It is also to be understood that any reference to any component, data, or structure in the embodiments of the disclosure, may be generally understood as one or more, unless explicitly defined otherwise or stated otherwise.

In addition, the term "and/or" in the present disclosure is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the former and latter associated objects are in an "or" relationship.

It should also be understood that the description of the various embodiments of the present disclosure emphasizes the differences between the various embodiments, and the same or similar parts may be referred to each other, so that the descriptions thereof are omitted for brevity.

Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

The disclosed embodiments may be applied to electronic devices such as terminal devices, computer systems, servers, etc., which are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known terminal devices, computing systems, environments, and/or configurations that may be suitable for use with electronic devices, such as terminal devices, computer systems, servers, and the like, include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set top boxes, programmable consumer electronics, network pcs, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above systems, and the like.

Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

Exemplary method

Fig. 1 is a schematic flow chart of a three-dimensional house model quality evaluation method according to an exemplary embodiment of the present disclosure. The method shown in fig. 1 comprises step 101, step 102 and step 103, which are described below.

Step 101, obtaining N two-dimensional house images corresponding to N visual angles according to a three-dimensional house model; wherein N is an integer greater than or equal to 2.

Here, a three-dimensional house model to be subjected to model quality evaluation may be obtained first, and then the three-dimensional house model may be processed by using a screenshot tool or a three-to-two conversion tool to obtain N two-dimensional house images corresponding to N viewing angles. Alternatively, N views and N two-dimensional house images may have a one-to-one correspondence, and N may have a value of 2, 5, 10, 15, 17, 20, etc., which are not listed herein.

And 102, inputting the N two-dimensional house images into a neural network to obtain N model quality evaluation data which are output by the neural network and correspond to the N two-dimensional house images.

Here, the neural network may be trained in advance, and the neural network may be a convolutional neural network such as incopytionresnetv 2.

In one embodiment, the neural network may be trained using a sample two-dimensional house image obtained from a sample three-dimensional house model, and model quality annotation data for the sample two-dimensional house image.

Here, a large number of sample three-dimensional house models may be obtained in advance, and each sample three-dimensional house model may be processed using a screenshot tool or a three-to-two conversion tool to obtain a large number of sample two-dimensional house images. Thereafter, labeling can be performed by the user for each sample two-dimensional house image to obtain model quality labeling data for each sample two-dimensional house image.

As can be seen from the investigation, the quality characteristics of the three-dimensional house model of interest to the user are shown in table 1 below.

TABLE 1

By summarizing the contents in table 1, model quality assessment dimensions to be annotated and assessment grades under each model quality assessment dimension can be made when a sample two-dimensional house image is annotated, and the following table 2 can be specifically seen.

TABLE 2

Therefore, when any sample two-dimensional house image is labeled, 7 model quality evaluation dimensions related to the table 2 can be respectively selected from all possible evaluation grades for labeling, so that 7 evaluation grades are always labeled for the sample two-dimensional house image, and model quality labeling data of the sample two-dimensional house image can be obtained based on the 7 evaluation grades. In one specific example, a certain model quality annotation datum may include the following information: integrity of modeling: the method is good; fine degree of mapping: the method is good; hole breaking degree: is very good; oblique twisting: is very good; color deviation: generally; model overlapping: good; shielding the model: good results are obtained.

Then, training may be performed according to a large number of sample two-dimensional house images, a large number of model quality labeling data, and a preset neural network training algorithm to obtain the neural network involved in step 102, where the obtained neural network can output corresponding model quality assessment data according to any input two-dimensional house image, and the output model quality assessment data may include assessment information (which may be specifically a probability group in the following) corresponding to each of the 7 model quality assessment dimensions involved in table 2.

After the neural network is trained, the trained neural network may be stored locally. In step 102, the N two-dimensional house images may be respectively input to the neural network, and at this time, the neural network may output N model quality evaluation data corresponding to the N two-dimensional house images, and there may be a one-to-one correspondence relationship between the N two-dimensional house images and the N model quality evaluation data.

And 103, outputting a model quality evaluation result of the three-dimensional house model according to the N visual angles and the N model quality evaluation data.

After the N pieces of model quality evaluation data are obtained, a model quality evaluation result of the three-dimensional house model can be determined by combining the N visual angles and the N pieces of model quality evaluation data, and the model quality evaluation result can be used for representing the overall quality of the three-dimensional house model and the quality of the three-dimensional house model under different visual angles and/or different model quality evaluation dimensions. And then, the model quality evaluation result can be output in the forms of screen display, mail sending, voice broadcasting and the like, so that relevant personnel can conveniently look up the model quality evaluation result.

On the basis of the embodiment shown in fig. 1, as shown in fig. 2, step 103 includes:

step 1031, obtaining M probability groups corresponding to M model quality evaluation dimensions for each two-dimensional house image in the N two-dimensional house images according to the N model quality evaluation data; wherein, the probability group corresponding to any model quality evaluation dimension comprises: the probability of each corresponding evaluation grade under the quality evaluation dimension of the model is M, which is an integer greater than or equal to 1;

step 1032, determining M evaluation scores corresponding to M model quality evaluation dimensions for each two-dimensional house image according to the M probability groups obtained for each two-dimensional house image;

and 1033, outputting a model quality evaluation result of the three-dimensional house model according to the N visual angles and the M evaluation scores determined for each two-dimensional house image.

Here, M may be 1, 5, 7, 10, or other values, which are not individually set forth herein. Corresponding to table 2 above, the value of M may specifically be 7, and the 7 model quality evaluation dimensions may be: modeling integrity, chartlet fineness, hole breaking degree, inclined distortion, color deviation, model overlapping and model shielding.

In an embodiment of the present disclosure, the model quality evaluation data corresponding to any two-dimensional house image may include: the two-dimensional house image is subjected to M probability groups corresponding to M model quality evaluation dimensions, so that in step 1031, M probability groups corresponding to M model quality evaluation dimensions can be directly obtained for the two-dimensional house image from model quality evaluation data corresponding to the two-dimensional house image, and in this way, M probability groups can be obtained for each two-dimensional house image in the N two-dimensional house images.

Next, step 1032 may be performed. In one embodiment, step 1032 may include:

obtaining weights corresponding to all evaluation grades under the quality evaluation dimension of the target model; the target model quality evaluation dimension is any one of M model quality evaluation dimensions;

and taking the weighted sum result as an evaluation score which is determined for the target two-dimensional house image and corresponds to the target model quality evaluation dimension.

Assuming that any probability in M probability groups determined for the target two-dimensional house image and in the probability group corresponding to the target model quality evaluation dimension is represented as P_iThe weight corresponding to any evaluation grade under the quality evaluation dimension of the target model is represented as S_iThe weighted SUM result SUM can be adoptedThe following formula is calculated:

in a specific example, if the target model quality evaluation dimension is the modeling integrity in table 2 above, the probability group corresponding to the target model quality evaluation dimension in the M probability groups determined for the target two-dimensional house image will include 5 probabilities corresponding to 5 evaluation levels under the modeling integrity, which are the probabilities P corresponding to "very poor" respectively₁The probability P corresponding to "difference₂The probability P of "general" correspondence₃Probability of "good" correspondence P₄The probability P of "very good" correspondence₅。

Next, 5 weights corresponding to 5 evaluation levels under modeling integrity, which are set in advance, may be obtained, each being a weight S corresponding to "very poor"₁The weight S corresponding to the "difference₂The weight S corresponding to "general₃The weight S corresponding to "good₄The weight S corresponding to "very good₅. Alternatively, S₁Can be 0, S₂May be 25, S₃Can be 50, S₄Can be 75, S₅May be 100.

Then, the weighted SUM result SUM can be obtained by using the following formula:

SUM＝S₁×P₁+S₂×P₂+S₃×P₃+S₄×P₄+S₅×P₅

then, the obtained SUM may be used as an evaluation score corresponding to the modeling integrity determined for the target two-dimensional house image.

It should be noted that, in the case that the target model quality evaluation dimension is a model quality evaluation dimension different from the model integrity in table 2 above, the calculation manner of the evaluation score is substantially similar to the above calculation manner, except that, in the case that the target model quality evaluation dimension is color deviation or model occlusion, 3 weights corresponding to the lower 3 evaluation dimensions (i.e., "severe," "general," and "good") may be 0, 50, and 100, respectively, and in the case that the target model quality evaluation dimension is model overlap, 4 weights corresponding to the lower 4 evaluation dimensions (i.e., "very severe," "general," and "good") may be 0, 33.3, 66.7, and 100, respectively.

In this embodiment, the evaluation score can be obtained easily and reliably by performing weighted sum calculation of the probability based on the weight.

Of course, the calculation of the evaluation score is not limited thereto, for example, in the case of P-based₁To P₅And S₁To S₅After the SUM is obtained, the SUM may be mapped to a designated score interval, e.g., 0-10, and the obtained mapping value is used as an evaluation score corresponding to the modeling integrity determined for the target two-dimensional house image.

After determining the M evaluation scores for each two-dimensional house image, a model quality evaluation result of the three-dimensional house model may be output based on the N view angles and the M evaluation scores determined for each two-dimensional house image. In one embodiment, the model quality assessment result may include: and each view angle in the N view angles and the mapping relation between the M evaluation scores determined for the two-dimensional house image corresponding to each view angle in the N two-dimensional house images.

Assuming that N values are 17, by executing the above steps, 7 evaluation scores corresponding to 7 model quality evaluation dimensions are respectively determined for each two-dimensional house image in 17 two-dimensional house images corresponding to 17 views, and it can be considered that 17 groups of scores corresponding to 17 two-dimensional house images are obtained, each group of scores includes 7 evaluation scores corresponding to the corresponding model quality evaluation dimensions, and then, a model quality evaluation result including a mapping relationship between 17 views and 17 groups of scores can be generated and output, and based on the mapping relationship in the model quality evaluation result, related personnel can conveniently and reliably know the quality of the three-dimensional house model at different views and different model quality evaluation dimensions.

Of course, the information included in the model quality evaluation result is not limited to this, for example, the calculation may be performed according to the N viewing angles and the M evaluation scores determined for each two-dimensional house image to obtain an evaluation total score, and the model quality evaluation result including the evaluation total score is generated and output, so that the relevant person can conveniently and reliably know the overall quality of the three-dimensional house model based on the total score of the model quality evaluation result.

In the embodiment of the disclosure, M probability groups can be obtained for each two-dimensional house image according to N pieces of model quality evaluation data, and M evaluation scores are further determined for each two-dimensional house image, so that the quality of the three-dimensional house model at different viewing angles and different model quality evaluation dimensions can be effectively represented according to the model quality evaluation results output from the N viewing angles and the M evaluation scores determined for each two-dimensional house image.

On the basis of the embodiment shown in fig. 1, as shown in fig. 3, step 101 includes:

step 1011, displaying the three-dimensional house model with a black background;

and 1012, performing screenshot processing on the three-dimensional house model displayed by the black background from the N visual angles to obtain N two-dimensional house images corresponding to the N visual angles.

It should be noted that, the three-dimensional house model generally has the characteristics that a window is exposed to form a white block, and a broken hole is a black block, and the three-dimensional house model rarely has a pure black part, if the three-dimensional house model is displayed by a white background, an exposure area of the window or an area with strong light can be pure white, at this time, the broken hole area and the exposure area can be confused, which easily affects the model quality evaluation effect of the three-dimensional house model, and if the three-dimensional house model is displayed by a black background, a black part appearing under the black background is a broken hole, which is beneficial to ensuring the model quality evaluation effect.

In view of this, in the embodiment of the present disclosure, for the three-dimensional house model to be subjected to model quality evaluation, the three-dimensional house model may be displayed in a black background, so as to avoid confusion between a hole-breaking area and an exposure area. Then, by means of the world coordinate system, N different viewing angles may be determined, and the different viewing angles may be controlled by horizontal and vertical yaw angles, and the 17 viewing angles may specifically include: the viewing angles expressed as (2 pi, 0), 8 viewing angles (0, 0), (0, pi/4), (0, 2 pi/4), (0, 3 pi/4), (0, pi/4), (0, 6 pi/4), (0, 7 pi/4) obtained by one cycle of horizontal viewing angle, and 8 viewing angles (pi/4, 0), (pi/4 ), (pi/4, 2 pi/4), (pi/4, 3 pi/4), (pi/4, 5 pi/4), (pi/4, 6 pi/4), (pi/4, 7 pi/4) obtained by one cycle of vertical upward pi/4. The three-dimensional house model displayed with the black background is subjected to screenshot processing from N different visual angles, N two-dimensional house images corresponding to the N different visual angles can be obtained, and model quality evaluation can be carried out by utilizing the N two-dimensional house images, so that the model quality evaluation effect can be well guaranteed.

In an alternative example, the inputting N two-dimensional house images into the neural network in step 102 includes:

It should be noted that there may be a large number of black areas at the edges of the two-dimensional house image obtained by performing the screen capture process on the three-dimensional house model displayed on the black background, as shown in fig. 4, for example, so that after N two-dimensional house images are obtained by the screen capture process, the edge-invalid black areas may be determined for each two-dimensional house image. Specifically, the coordinates of the upper, lower, left, and right edge points of the two-dimensional house image in fig. 4 may be determined first, and then, the two-dimensional house image may be gradually searched for a region which is closer to any edge point and includes pixel values of all pixel points that are pixel values corresponding to black (for example, 0), any region that is found may be an edge invalid black region, and the edge invalid black region may be considered as a region having no reference value when performing model quality evaluation.

And then, the edge invalid black area in each two-dimensional house image in the N two-dimensional house images can be removed, and the N two-dimensional house images with the edge invalid black areas removed are input into the neural network, so that the influence of confusion of a hole breaking area and an exposure area on the model quality evaluation effect can be avoided, the size of the two-dimensional house image input into the neural network can be reduced, the time required by model quality evaluation can be favorably shortened, and the model quality evaluation efficiency can be improved.

In an alternative example, a three-dimensional house model may also be referred to as a house three-dimensional (VR). In order to realize the model quality evaluation of the house three-dimensional VR, the following steps can be taken:

the method comprises the following steps: simulating an attention visual angle of human eyes to the three-dimensional VR of the house, carrying out front-end display on the three-dimensional VR of the sample house, and taking screenshots at different visual angles to obtain a two-dimensional image set of the three-dimensional VR of the sample house, wherein the two-dimensional image set comprises a plurality of screenshots (equivalent to the above two-dimensional house image) of the house;

step two: marking house three-dimensional VR screenshots in the two-dimensional picture set of the sample house three-dimensional VR obtained in the first step according to the 7 model quality evaluation dimensions related in the table 2;

step three: training by taking the house three-dimensional VR screenshot in the two-dimensional picture set of the sample house three-dimensional VR obtained in the step one and the labeling result (equivalent to the model quality labeling data) obtained in the step two as the input of a convolutional neural network;

step four: and integrating the house three-dimensional VR evaluation flow into a house three-dimensional VR production flow to obtain a model quality evaluation result of each house three-dimensional VR.

Alternatively, since table 2 mentioned above refers to a total of 30 evaluation levels in 7 model quality evaluation dimensions, in step three mentioned above, as shown in fig. 5, the output neurons in the last layer of the convolutional neural network may be set to be 30, in the convolutional neural network, a softmax activation function may be added according to the number of evaluation levels in each model quality evaluation dimension, and cross entropy is calculated, so that the trained convolutional neural network can output 7 probability groups (i.e., probability group 1 to probability group 7) corresponding to 7 model quality evaluation dimensions for any input image (i.e., the above two-dimensional house image), and the probability group corresponding to any model quality evaluation dimension may be regarded as the probability distribution of each evaluation level in the model quality evaluation dimension.

Optionally, as shown in fig. 6, after the house three-dimensional VR evaluation flow is integrated into the house three-dimensional VR production flow, the specific implementation process may be:

(1) shooting a house, uploading shooting data, and performing 3-dimensional reconstruction according to the shooting data to obtain a three-dimensional VR of the house;

(2) performing angle rotation based on the house three-dimensional VR, and capturing a picture at a plurality of viewing angles to obtain captured picture data (equivalent to N two-dimensional house images corresponding to the N viewing angles in the above description); in order to realize screenshot, a Chrome kernel five frame can be called to open a house three-dimensional VR, a default pitch angle is kept unchanged, different visual angles are realized by controlling horizontal and vertical yaw angles, and screenshot is realized by means of a Chrome screenshot function;

(3) storing the house id and the screenshot data to a storage structure;

(4) if the quality of the house three-dimensional VR model is to be evaluated, corresponding screenshot data can be downloaded, and the trained convolutional neural network is called to obtain the output result of the convolutional neural network (which is equivalent to the N pieces of model quality evaluation data in the above);

(5) the output result is stored in a lightweight data interchange format (JSON), and meanwhile, a score (equivalent to the above calculation evaluation score) can be calculated according to a preset weight, and then, a model quality evaluation result can be output, so that model quality evaluation is realized.

In summary, the embodiment of the disclosure can unify the evaluation systems of the house three-dimensional VRs obtained by different acquisition devices and different reconstruction algorithms under the condition that no standard data set is used as a reference, and meanwhile, the model quality evaluation results conform to the subjective perception of the user in each model quality evaluation dimension, and the model quality evaluation results have higher consistency with the model quality evaluation results of artificial subjectivity and have stable performance.

Any of the three-dimensional house model quality assessment methods provided by embodiments of the present disclosure may be performed by any suitable device having data processing capabilities, including but not limited to: terminal equipment, a server and the like. Alternatively, any of the three-dimensional house model quality assessment methods provided by the embodiments of the present disclosure may be executed by a processor, for example, the processor may execute any of the three-dimensional house model quality assessment methods mentioned in the embodiments of the present disclosure by calling a corresponding instruction stored in a memory. And will not be described in detail below.

Exemplary devices

Fig. 7 is a schematic structural diagram of a three-dimensional house model quality assessment apparatus according to an exemplary embodiment of the present disclosure, and the apparatus shown in fig. 7 includes a first obtaining module 701, a second obtaining module 702, and an output module 703.

A first obtaining module 701, configured to obtain N two-dimensional house images corresponding to N viewing angles according to a three-dimensional house model; wherein N is an integer greater than or equal to 2;

a second obtaining module 702, configured to input the N two-dimensional house images into a neural network, so as to obtain N pieces of model quality evaluation data output by the neural network and corresponding to the N two-dimensional house images;

and the output module 703 is configured to output a model quality evaluation result of the three-dimensional house model according to the N viewing angles and the N model quality evaluation data.

In an alternative example, the output module 703 includes:

the determining submodule is used for determining M evaluation scores corresponding to M model quality evaluation dimensions for each two-dimensional house image according to M probability groups obtained for each two-dimensional house image;

and the output submodule is used for outputting a model quality evaluation result of the three-dimensional house model according to the N visual angles and the M evaluation scores determined for each two-dimensional house image.

In one optional example, the determining sub-module comprises:

the first acquisition unit is used for acquiring weights corresponding to all evaluation grades under the quality evaluation dimensionality of the target model; the target model quality evaluation dimension is any one of M model quality evaluation dimensions;

the second acquisition unit is used for carrying out weighted summation on all probabilities in the M probability groups determined for the target two-dimensional house image and in the probability group corresponding to the target model quality evaluation dimension according to the obtained weights so as to obtain a weighted summation result; the target two-dimensional house image is any one of the N two-dimensional house images;

In one optional example, the model quality assessment results include: and each view angle in the N view angles and the mapping relation between the M evaluation scores determined for the two-dimensional house image corresponding to each view angle in the N two-dimensional house images.

In an optional example, the first obtaining module 701 includes:

In an optional example, the second obtaining module 702 includes:

and the output unit is used for inputting the N two-dimensional house images with the edge invalid black areas removed into the neural network.

In an alternative example, the neural network is trained using a sample two-dimensional house image obtained from a sample three-dimensional house model, and model quality annotation data for the sample two-dimensional house image.

Exemplary electronic device

Next, an electronic apparatus according to an embodiment of the present disclosure is described with reference to fig. 8. The electronic device may be either or both of the first device and the second device, or a stand-alone device separate from them, which stand-alone device may communicate with the first device and the second device to receive the acquired input signals therefrom.

Fig. 8 illustrates a block diagram of an electronic device 800 in accordance with an embodiment of the disclosure.

As shown in fig. 8, an electronic device 800 includes one or more processors 801 and memory 802.

The processor 801 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 800 to perform desired functions.

Memory 802 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by the processor 801 to implement the three-dimensional house model quality assessment methods of the various embodiments of the present disclosure described above and/or other desired functions. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.

In one example, the electronic device 800 may further include: an input device 803 and an output device 804, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

For example, when the electronic device 800 is a first device or a second device, the input means 803 may be a microphone or a microphone array. When the electronic device 800 is a stand-alone device, the input means 803 may be a communication network connector for receiving the acquired input signals from the first device and the second device.

The input device 803 may also include, for example, a keyboard, a mouse, and the like.

The output device 804 may output various information to the outside. The output devices 804 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.

Of course, for simplicity, only some of the components of the electronic device 800 relevant to the present disclosure are shown in fig. 8, omitting components such as buses, input/output interfaces, and the like. In addition, electronic device 800 may include any other suitable components depending on the particular application.

Exemplary computer program product and computer-readable storage Medium

In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the three-dimensional house model quality assessment method according to various embodiments of the present disclosure described in the "exemplary methods" section above of this specification.

The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps in the three-dimensional house model quality assessment method according to various embodiments of the present disclosure described in the "exemplary methods" section above in this specification.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.

In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts in the embodiments are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

The block diagrams of devices, apparatuses, systems referred to in this disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".

The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

It is also noted that in the devices, apparatuses, and methods of the present disclosure, each component or step can be decomposed and/or recombined. These decompositions and/or recombinations are to be considered equivalents of the present disclosure.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims

1. A three-dimensional house model quality assessment method is characterized by comprising the following steps:

2. The method of claim 1, wherein determining M evaluation scores for each two-dimensional house image for the M model quality evaluation dimensions based on the M probability sets obtained for each two-dimensional house image comprises:

and taking the weighted summation result as an evaluation score which is determined by the target two-dimensional house image and corresponds to the target model quality evaluation dimension.

3. The method of claim 1, wherein the model quality assessment results comprise: and each view angle in the N view angles and the mapping relation between the M evaluation scores determined for the two-dimensional house image corresponding to each view angle in the N two-dimensional house images.

4. The method of claim 1, wherein obtaining N two-dimensional house images corresponding to N views from the three-dimensional house model comprises:

displaying the three-dimensional house model with a black background;

5. The method of claim 4, wherein said inputting said N two-dimensional house images into a neural network comprises:

6. The method of claim 1, wherein the neural network is trained using a sample two-dimensional house image obtained from a sample three-dimensional house model, and model quality annotation data for the sample two-dimensional house image.

7. A three-dimensional house model quality evaluation device is characterized by comprising:

the output module is used for outputting a model quality evaluation result of the three-dimensional house model according to the N visual angles and the N model quality evaluation data;

the output module includes:

8. The apparatus of claim 7, wherein the determining sub-module comprises:

and the determining unit is used for taking the weighted summation result as an evaluation score which is determined by the target two-dimensional house image and corresponds to the target model quality evaluation dimension.

9. The apparatus of claim 7, wherein the model quality assessment results comprise: and each view angle in the N view angles and the mapping relation between the M evaluation scores determined for the two-dimensional house image corresponding to each view angle in the N two-dimensional house images.

10. The apparatus of claim 7, wherein the first obtaining module comprises:

11. The apparatus of claim 10, wherein the second obtaining module comprises:

12. The apparatus of claim 7, wherein the neural network is trained using a sample two-dimensional house image obtained from a sample three-dimensional house model, and model quality annotation data for the sample two-dimensional house image.

13. A computer-readable storage medium storing a computer program for executing the three-dimensional house model quality assessment method according to any one of claims 1 to 6.

14. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

the processor is used for reading the executable instructions from the memory and executing the instructions to realize the three-dimensional house model quality evaluation method of any one of the claims 1 to 6.