CN113298928A - House three-dimensional reconstruction method, device, equipment and storage medium - Google Patents
House three-dimensional reconstruction method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN113298928A CN113298928A CN202110327798.0A CN202110327798A CN113298928A CN 113298928 A CN113298928 A CN 113298928A CN 202110327798 A CN202110327798 A CN 202110327798A CN 113298928 A CN113298928 A CN 113298928A
- Authority
- CN
- China
- Prior art keywords
- shooting
- panoramic
- panoramic image
- room
- house
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000010586 diagram Methods 0.000 claims description 43
- 238000013507 mapping Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 description 9
- 238000010276 construction Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
Abstract
The embodiment of the invention provides a house three-dimensional reconstruction method, a device, equipment and a storage medium, wherein the house three-dimensional reconstruction method comprises the following steps: acquiring a first panoramic view and a house type view corresponding to a room; determining the shooting orientation of the first panoramic image according to the preset orientation information corresponding to the house type image; determining the shooting position of the first panoramic image according to the shooting orientation of the first panoramic image and the house type image; and according to the shooting orientation and the shooting position of the first panoramic image, performing three-dimensional reconstruction on the room. The shooting position and the shooting orientation of the panoramic image are determined based on the prior house type image, and the method is simple and accurate to realize. The accurate reconstruction of indoor scenes of rooms can be realized by combining the prior house type graph.
Description
Technical Field
The invention relates to the technical field of internet, in particular to a house three-dimensional reconstruction method, a device, equipment and a storage medium.
Background
Such as in a scenario where the user rents or purchases a house, it may be inconvenient for the user to look at the house in the field. In order to enable the user to view the house online, the house may be subjected to three-dimensional scene reconstruction in advance to obtain a 3D model of the house, which is provided for the user to view.
How to efficiently and accurately complete the three-dimensional reconstruction task of the house is an urgent problem to be solved.
Disclosure of Invention
The embodiment of the invention provides a house three-dimensional reconstruction method, a house three-dimensional reconstruction device, house three-dimensional reconstruction equipment and a storage medium, which can conveniently and accurately complete the house three-dimensional reconstruction.
In a first aspect, an embodiment of the present invention provides a house three-dimensional reconstruction method, where the method includes:
acquiring a first panoramic view and a house type view corresponding to a room;
determining the shooting orientation of the first panoramic image according to preset orientation information corresponding to the house type image;
determining the shooting position of the first panoramic image according to the shooting orientation of the first panoramic image and the house type image;
and performing three-dimensional reconstruction on the room according to the shooting orientation and the shooting position of the first panoramic image.
In a second aspect, an embodiment of the present invention provides a house three-dimensional reconstruction apparatus, including:
the acquisition module is used for acquiring a first panoramic image and a house type image corresponding to a room;
the pose determining module is used for determining the shooting direction of the first panoramic image according to preset direction information corresponding to the house type image; determining the shooting position of the first panoramic image according to the shooting orientation of the first panoramic image and the house type image;
and the reconstruction module is used for performing three-dimensional reconstruction on the room according to the shooting direction and the shooting position of the first panoramic image.
In a third aspect, an embodiment of the present invention provides an electronic device, which includes a processor and a memory, where the memory stores executable codes, and when the executable codes are executed by the processor, the processor may implement at least the house three-dimensional reconstruction method in the first aspect.
In a fourth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored, and when the executable code is executed by a processor of an electronic device, the processor is enabled to implement at least the house three-dimensional reconstruction method in the first aspect.
In the process of three-dimensional reconstruction of a house comprising one or more rooms, three-dimensional reconstruction is carried out on each room to obtain a three-dimensional model of each room, and then the three-dimensional models of different rooms are spliced together according to the adjacency relation between the rooms to obtain the three-dimensional model of the whole house.
In the process of three-dimensional reconstruction of a room, a house type map of the room and a panorama (referred to as a first panorama) taken in the room are used to complete the three-dimensional reconstruction processing of the room based on the house type map and the first panorama. Specifically, to accomplish the three-dimensional reconstruction of the room, it is necessary to know pose information of the camera at the time of photographing the first panorama, in order to accomplish the three-dimensional reconstruction of the room based on the pose information, wherein the pose information includes a photographing orientation and a photographing position, the photographing position and the photographing orientation being with respect to the house type map. The coordinates corresponding to the position of the user when the user takes the first panorama in the house type map are referred to as a shooting position, and the shooting orientation is the orientation angle of the camera facing the mark in the house type map when the first panorama is taken.
Specifically, the shooting orientation of the first panoramic image may be determined according to the preset orientation information corresponding to the house type map of the room, and then the shooting position of the first panoramic image may be determined according to the shooting orientation of the first panoramic image and the house type map of the room. The shooting position and the shooting orientation of the panoramic image are determined based on the prior house type image, and the method is simple and accurate to realize. The accurate reconstruction of indoor scenes of rooms can be realized by combining the prior house type graph.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is a flowchart of a house three-dimensional reconstruction method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a house view according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a result of determining a shooting orientation according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a result of determining a shooting position according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a result of three-dimensional house reconstruction according to an embodiment of the present invention;
fig. 6 is a flowchart of a house three-dimensional reconstruction method according to another embodiment of the present invention;
fig. 7 is a schematic diagram of a result of three-dimensional house reconstruction according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of a house three-dimensional reconstruction apparatus according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an electronic device corresponding to the three-dimensional house reconstruction apparatus provided in the embodiment shown in fig. 8.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terminology used in the embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the examples of the present invention and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise, and "a plurality" typically includes at least two.
The words "if", as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination" or "in response to a detection", depending on the context. Similarly, the phrases "if determined" or "if detected (a stated condition or event)" may be interpreted as "when determined" or "in response to a determination" or "when detected (a stated condition or event)" or "in response to a detection (a stated condition or event)", depending on the context.
The house three-dimensional reconstruction method provided by the embodiment of the invention can be executed by an electronic device. The electronic device may be a user terminal device such as a camera, a mobile phone, a tablet computer, a computer, or the like. The electronic device can also be a server or a virtual machine provided by the cloud.
The house three-dimensional reconstruction method provided by the embodiment of the invention is used for realizing the three-dimensional reconstruction of the indoor scene of the house so as to construct the corresponding three-dimensional model. The house may be a house, a factory building, an office, etc.
In practical applications, a house often has at least one room, for example, a house has a plurality of functional spaces such as a master bedroom, a secondary bedroom, a bathroom, a kitchen, a living room, and the like, and each space can be regarded as a room, and three-dimensional reconstruction is performed on the house, that is, three-dimensional reconstruction needs to be performed on each room in the house, so as to obtain a three-dimensional model corresponding to each room. Then, based on the adjacent relation between the rooms, the three-dimensional models of different rooms can be spliced together according to the adjacent relation between the rooms, and the three-dimensional model of the whole house can be obtained.
Since the principles of the three-dimensional reconstruction process for each room are similar, the three-dimensional reconstruction process for any room in the house is mainly described in the following embodiments.
Fig. 1 is a flowchart of a house three-dimensional reconstruction method according to an embodiment of the present invention, as shown in fig. 1, which may include the following steps:
101. and acquiring a first panoramic view and a house type view corresponding to the room.
102. And determining the shooting direction of the first panoramic image according to the preset direction information corresponding to the house type image.
103. And determining the shooting position of the first panoramic image according to the shooting orientation of the first panoramic image and the house type image.
104. And according to the shooting orientation and the shooting position of the first panoramic image, performing three-dimensional reconstruction on the room.
The room in this embodiment may be any room in a house. The scheme provided by the embodiment is used for realizing three-dimensional reconstruction of the room.
In the solution provided by this embodiment, a priori (known) house type map of the room needs to be combined to assist the three-dimensional reconstruction of the room.
The house type graph of the room can be analyzed from the whole house type graph of the house. The overall floor plan of the house can have the identification of each room and the boundary line mark corresponding to each room in the overall floor plan, and the floor plan corresponding to each room can be separated from the overall floor plan based on the identification of each room.
For ease of understanding, the description is illustrative with reference to FIG. 2.
In fig. 2, it is assumed that a house includes two rooms, an overall floor plan of the house is shown in fig. 2, two room identifiers, namely "room 1" and "room 2", are provided in the overall floor plan, a region corresponding to "room 1" is cut out from the overall floor plan and is a floor plan corresponding to room 1, a region corresponding to "room 2" is cut out from the overall floor plan and is a floor plan corresponding to room 2, and the cut-out result is shown in fig. 2.
The room identifier may be a text identifier or a graphic identifier. The character identification can be characters such as 'main lying', 'secondary lying', 'toilet', 'kitchen', and the like. The graphic identifier may be represented by an object graphic reflecting the main function of each room, such as a toilet graphic corresponding to a toilet, a bed graphic corresponding to a primary lying and a secondary lying, and a kitchen graphic corresponding to a kitchen range.
In practical applications, the overall house type map of the house may be collected by a user (the user herein may refer to a person who performs a three-dimensional house reconstruction task) in advance. The analysis operation of the house type diagram corresponding to each room may be implemented by a manual operation of a user, for example, the user performs a clipping operation on the entire house type diagram according to the boundary line corresponding to each room to obtain the house type diagram corresponding to each room; or, the user marks the corner points corresponding to each room, determines the coordinates corresponding to each corner point, and connects the coordinates of the corner points in sequence to obtain the house type graph of the room. The analysis operation may also be automatically identified based on image features, for example, a room identifier corresponding to each room and a specific line indicating a boundary line of the room are identified in the overall floor plan, so as to segment the floor plan corresponding to each room.
As shown in fig. 2, the user may also perform calibration of preset orientation information on the overall house type diagram, where the preset orientation information may be represented by four orientation angles, i.e. 0 degrees, 90 degrees, 180 degrees and 270 degrees, and the four orientation angles correspond to four directions corresponding to houses in the house type diagram: east, west, south, north. For example, if the user defines "north" as 0 degrees, then "east" corresponds to 90 degrees, "south" corresponds to 180 degrees, and "west" corresponds to 270 degrees. The calibration of the four orientation angles in the whole house type image provides a unified reference standard for determining shooting orientations of different panoramic images.
It can be understood that, after the preset orientation information corresponding to the whole house type diagram is set, the preset orientation information of the house type diagram of each room is also determined, and is consistent with the whole house type diagram, as shown in fig. 2.
For obtaining the panoramic image, in order to perform three-dimensional reconstruction of a house, in a house including at least one room, a user may take one or more panoramic images in each room, and may name each panoramic image by a room identifier to distinguish rooms corresponding to different panoramic images. Thus, based on the room identifier corresponding to each panoramic image and the room identifier corresponding to the house type image of each room as described above, the corresponding relationship between the panoramic image and the house type image of the room can be known. Therefore, three-dimensional reconstruction of a room can be realized based on the house type picture corresponding to the room and the panoramic picture shot in the room.
For a room, a user may stand at different locations (i.e., different shooting locations) in the room to shoot multiple panoramas in different shooting orientations, or the user may shoot one panoramas in a certain shooting orientation at a certain shooting location in the room. The first panorama of the room in step 101 may be any panorama taken by the user in the room.
It will be appreciated that during the taking of a panorama, the user may place the camera on a fixed height stand and then rotate the camera 360 degrees from an initial shooting orientation to take a panorama. The shooting orientation in this context means the initial shooting orientation.
When a user takes a panoramic image (such as a first panoramic image) in a room, the user does not know the pose information of the camera in the room, which includes the shooting direction and shooting position of the camera, and needs to know the pose information of the camera when the user takes the first panoramic image, namely the shooting direction and shooting position of the first panoramic image, in order to perform three-dimensional reconstruction of the room.
The shooting position here refers to the position of the horizontal and vertical coordinates, and the height position may be known, for example, the height position is a fixed height of the tripod.
The shooting orientation refers to which direction of the room the initial orientation of the camera was facing when the first panorama was shot. As can be seen from the above, the four orientations of the room are respectively expressed by the preset four angular values, and therefore, determining the shooting orientation of the first panorama is to determine which of the four angles the camera is actually facing when shooting the first panorama.
In the embodiment of the invention, the shooting position and the shooting orientation of the first panoramic picture can be determined by means of the prior house type picture of the room.
First, the shooting orientation of the first panorama can be determined according to preset orientation information corresponding to a house type map of a room.
Specifically, the first panorama can be projected toward the ground to obtain a bottom view corresponding to the first panorama, which is a two-dimensional image. And further, determining the shooting orientation of the first panoramic image according to the structural characteristics and the preset orientation information presented in the house type image of the room and the structural characteristics presented in the bottom image. The bottom view corresponding to the first panoramic image actually shot and the structural features respectively shown in the house type image are compared with the orientation angle corresponding to each direction marked in the house type image, so that the shooting orientation of the first panoramic image can be known.
For ease of understanding, the description is exemplified in conjunction with fig. 3. It is assumed that a house type diagram of a room and preset orientation information are as shown in fig. 3, and that a bottom view corresponding to the first panorama is as shown in fig. 3. As can be seen from the illustration in fig. 3, the structural features of the bottom view are matched with those of the floor plan after rotating clockwise 90 degrees, i.e. the right front of the bottom view actually corresponds to the orientation marked as 90 degrees in the floor plan, so that the shooting orientation of the first panorama is determined to be 90 degrees.
After the shooting orientation of the first panoramic image is determined, the shooting position of the first panoramic image, namely the horizontal and vertical coordinate values of the user (camera) in the room when the first panoramic image is shot, can be determined based on the shooting orientation of the first panoramic image and the house type image of the room.
Alternatively, determining the shooting position of the first panorama can be implemented as:
determining a first structural drawing of the room from the first panorama;
carrying out direction adjustment on the first structural diagram according to the shooting orientation of the first panoramic diagram;
carrying out corner matching processing on the first structural drawing and the house-type drawing after the direction adjustment;
and determining the mapping coordinate corresponding to the shooting reference point in the first structural drawing in the house type drawing according to the corner matching result, wherein the mapping coordinate is used as the shooting position of the first panoramic drawing.
The first structural diagram is obtained by determining a structure (layout) of the first panorama, and specifically may be obtained through the following processes: firstly, a three-dimensional structure diagram of a room is determined through a first panoramic view, and then the three-dimensional structure diagram is mapped into a two-dimensional structure diagram, wherein the two-dimensional structure diagram is a first structure diagram.
In order to obtain the three-dimensional structure diagram, a wall line needs to be identified from the first panoramic view so as to construct the three-dimensional structure diagram based on the identified wall line. The identification of the wall line can be realized by referring to the prior related art, and is not described herein.
Because the first panoramic image needs to be processed by wall line identification and the like, in order to obtain a more accurate result, optionally, when the first panoramic image is shot, a user may label some key point information in the first panoramic image, such as marking out a wall surface/wall line, a roof, a ground, a door and the like in the first panoramic image, and the labeled information may assist in processing wall line identification, determination of connection relationships between different rooms (connected through doors) and the like.
Assuming that the shooting orientation of the first panorama is a 90-degree direction, the direction adjustment of the first configuration map is to rotate the first configuration map in the 90-degree direction so that the first configuration map matches with the structural features of the house type map of the room. And then carrying out corner matching on the first structural drawing after the direction adjustment and the house type drawing so as to determine the corner corresponding relation for matching the structural features of the first structural drawing and the house type drawing.
Assuming that the first structural drawing and the house type drawing after the direction adjustment are as shown in fig. 4, corner points corresponding to the corner points in the first structural drawing are sequentially determined in the house type drawing. The shooting reference point of the first structural diagram corresponds to a shooting position corresponding to when the user shoots the first panorama, and the central point of the first structural diagram can be used as the shooting reference point. Based on the relative position relationship of the center point of the first structural figure with respect to each corner point of the first structural figure, the corresponding relationship between the center point of the first structural figure and the corner point of the house type figure and the coordinates of each corner point in the house type figure, the corresponding mapping coordinates of the center point of the first structural figure in the house type figure can be determined, and finally the mapping coordinates are used as the shooting position of the first panoramic figure.
The above embodiment describes the correction of the first structural image corresponding to the first panorama based on the determined shooting orientation of the first panorama. In fact, optionally, the first panorama may be corrected based on the shooting orientation, and then the corrected first panorama is subjected to layout determination to obtain a first structural diagram, and the first structural diagram and the house type diagram are subjected to corner point matching to determine the shooting position of the first panorama. Here, the correction of the first panorama can be understood as performing rotation transformation on the first panorama according to an angle corresponding to a shooting orientation.
After the shooting position and the shooting orientation of the first panorama are obtained, three-dimensional reconstruction of the room can be performed. In brief, the height information of the room may be known in advance, an initial three-dimensional model (or a three-dimensional house type map) may be constructed based on the house type map and the height information of the room, then, the position points corresponding to the pixels in the first panoramic image are determined in the three-dimensional model according to the shooting position and the shooting orientation of the first panoramic image, and the pixels corresponding to the corresponding position points are "mapped" to the three-dimensional model, so that the three-dimensional reconstruction result including the real scene in the room may be obtained.
It will be appreciated that if the first panorama is understood to be composed of a plurality of consecutive frame images, the first frame image is the scene in a certain range of the shooting orientation at the shooting position in the room where the camera is facing, and then the camera is rotated to continue shooting the subsequent plurality of frame images. And positioning the shooting position and the shooting direction in the initial three-dimensional model, obtaining the shooting range corresponding to the first frame image, determining the position point corresponding to each pixel point in the first frame image in the range corresponding to the shooting direction, completing mapping of the first frame image, and then completing mapping of the subsequent other frame images based on the automatically recorded subsequent other frame images and the rotation angle of the camera shooting the first frame image.
The three-dimensional reconstruction processing is carried out on each room in one house, and three-dimensional models obtained by three-dimensional reconstruction of different rooms are spliced based on the connection relation among the rooms, so that the three-dimensional reconstruction result of the house can be obtained. Assuming that the obtained three-dimensional reconstruction result of a certain house is as shown in fig. 5, the indoor scene situation of the house can be seen through the three-dimensional model obtained through reconstruction.
In summary, in the three-dimensional house reconstruction scheme provided in the embodiment of the present invention, the user only needs to upload the house type map with the orientation information mark and acquire the panoramic view of the room, and thus, the manual operation is less and the automation degree is high. In addition, the panoramic image shooting orientation can be accurately determined based on the prior house type image and the panoramic image which are easy to obtain, the panoramic image shooting position can be accurately determined based on the layout determination result of the panoramic image and the house type image, and the panoramic image shooting orientation determination method and the panoramic image shooting position determination method are convenient and accurate to achieve.
Fig. 6 is a flowchart of a house three-dimensional reconstruction method according to another embodiment of the present invention, as shown in fig. 6, the method may include the following steps:
601. and acquiring a house type picture and a plurality of panoramic pictures corresponding to the room.
602. And for any panoramic picture in the plurality of panoramic pictures, determining the matching degree of the characteristic points between the panoramic picture and the rest panoramic pictures, and determining the first panoramic picture with the matching degree of the characteristic points meeting the set requirement according to the matching degree of the characteristic points corresponding to the plurality of panoramic pictures.
603. Determining the shooting orientation of the first panoramic picture according to the preset orientation information corresponding to the house type picture, determining the first structural picture of the room according to the first panoramic picture, and determining the shooting position of the first panoramic picture according to the first structural picture, the shooting orientation of the first panoramic picture and the house type picture.
604. And determining the shooting orientation and the shooting position of a second panoramic image in the plurality of panoramic images, wherein the second panoramic image is any panoramic image except the first panoramic image in the plurality of panoramic images.
605. And performing three-dimensional reconstruction on the room according to the shooting direction and the shooting position of the first panoramic image, and displaying the mark points corresponding to the shooting position of the second panoramic image in a three-dimensional model obtained through the three-dimensional reconstruction.
606. And responding to the selection operation of the mark point, and displaying the second panoramic image according to the shooting orientation of the second panoramic image.
The scheme provided by the embodiment can achieve two purposes: one is to carry out three-dimensional reconstruction on the house to obtain a three-dimensional model corresponding to the house; and secondly, the multi-view roaming viewing effect of the room is provided.
In order to realize the multi-view roaming viewing effect of a room, a user needs to shoot a plurality of panoramas in one room, and the shooting positions and shooting orientations of the plurality of panoramas can be different from each other. The multiple panoramas are taken in different orientations, meaning that the multiple panoramas can provide multiple different viewing angles to view the room.
Visually, for an object in a room, different shooting positions result in different distances from the object relative to the object, so that the pixel areas occupied by the object in different panoramas are different in size; different shooting orientations may cause the object to appear differently in different panoramas, and may even fail to see the object in a certain panorama due to occlusion.
For a room, three-dimensional reconstruction of the room can be realized based on one panoramic image taken in the room, and in addition, multi-view roaming viewing effect of the room is realized based on a plurality of panoramic images taken in the room.
In the aforementioned other embodiments, it is mentioned that when a plurality of panoramas are taken in a room, optionally one panoramas can be determined therefrom as the first panoramas at will, to realize a three-dimensional reconstruction of the room based on the first panoramas. In fact, the selection of the first panorama can also be implemented in other ways, such as the way provided in this embodiment: and (5) a characteristic point matching mode.
For example, assuming that a user takes 3 panoramas in one room, the 3 panoramas have different shooting positions and different shooting orientations, and the three panoramas are respectively represented as a panorama a, a panorama B, and a panorama C. And performing feature point matching on the panoramic image A and the panoramic image B, wherein 10 feature points of the panoramic image A and the panoramic image B are matched, and similarly, performing feature point matching on the panoramic image A and the panoramic image C, and wherein 15 feature points of the panoramic image A and the panoramic image C are matched. And similarly, respectively matching the panoramic image B with the panoramic image A and the panoramic image C for feature points to obtain the number of matched feature points corresponding to the panoramic image B and the two remaining panoramic images. The same processing is performed for the panorama C.
For each panorama, the matching degree of the feature points corresponding to the panorama can be determined according to the number of the feature points matched with the remaining other panoramas, for example, the minimum value, the maximum value or the average value of the number of the feature points matched with the remaining other panoramas is used as the matching degree of the feature points corresponding to the panorama. Assuming that the minimum value is taken as an example, in the above example, the matching degree of the feature points corresponding to the panorama a is 10.
And according to the matching degree of the feature points corresponding to the 3 panoramic pictures, selecting one panoramic picture meeting the set requirement as a first panoramic picture. For example, the setting requirement is that the feature point matching degree is highest.
Through the selection processing of the first panoramic image, the selected first panoramic image is often a panoramic image which is shot close to the center of the room and can better capture the indoor scene of the room.
After the first panoramic image is obtained, the shooting orientation and the shooting position of the first panoramic image are determined, and a process of three-dimensionally reconstructing a room according to the shooting orientation and the shooting position of the first panoramic image may refer to the related description in the foregoing other embodiments, which is not described herein again.
In order to obtain the multi-view roaming viewing effect of the room, for the remaining panoramic views except for the first panoramic view, the shooting orientations and shooting positions corresponding to the panoramic views also need to be determined, so that when the multi-view roaming viewing is performed, based on the shooting orientations and the shooting positions, the orientation information reflected by the corresponding panoramic views is consistent with the orientation information of the three-dimensional model obtained by three-dimensional reconstruction of the room, that is, consistent with the orientation information marked in the house type map.
Taking the second panorama (any of the remaining panoramas) as an example, the shooting orientation and shooting position of the second panorama can be determined in the following manner:
matching feature points of the second panoramic image with the first panoramic image to determine a rotation angle according to the matched feature points, and determining the shooting orientation of the second panoramic image according to the shooting orientation of the first panoramic image and the rotation angle;
and determining a second structure diagram of the room according to the second panoramic image, and determining the shooting position of the second panoramic image according to the second structure diagram, the shooting direction of the second panoramic image and the house type diagram of the room.
Assuming that the first panorama is shot in the direction a and the second panorama is shot in the direction b, for an object in the room, which is shot downward, the object may appear differently in the corresponding first and second panoramas. For example, if the two orientations are consistent, the morphological difference of the object in the two panoramas may appear as the size difference of the occupied pixel area; as another example, if the two orientations are not identical, the difference in the shape of the object in the two panoramas may appear to be different.
By matching the feature points in the two panoramas, a rotation angle can be determined based on the coordinates of the matched pairs of feature points in the corresponding panoramas, the rotation angle indicates the rotation information of the orientation b of the second panoramas relative to the orientation a of the first panoramas, for example, the rotation of 90 degrees clockwise is consistent with the orientation a, and then the orientation b can be known when the orientation a is known. For example, if the direction a is 0 degrees of the house type diagram and the rotation angle is 90 degrees clockwise, the direction b is 90 degrees.
After the shooting direction of the second panoramic image is obtained, the process of determining the shooting position of the second panoramic image according to the second structure diagram corresponding to the second panoramic image, the shooting direction of the second panoramic image and the house type diagram of the room is consistent with the process of determining the shooting position of the first panoramic image in the foregoing embodiment, and is not repeated.
Thus, the shooting directions and shooting positions corresponding to the plurality of panoramas shot in one room are obtained. And then, performing three-dimensional reconstruction on the room according to the shooting direction and the shooting position of the first panoramic image, and displaying the mark points corresponding to the shooting positions of the rest panoramic images in the three-dimensional model obtained through the three-dimensional reconstruction. The mark points are operation entries for users to switch and inquire the panoramic pictures shot at different visual angles, and after a certain mark point is selected by the user, the panoramic picture is displayed according to the shooting direction of the panoramic picture corresponding to the mark point.
This is illustrated in connection with fig. 7. Assuming that a three-dimensional model of a house reconstructed from first panoramas taken in respective rooms of the house is as shown in fig. 7, and assuming that three panoramas are taken in a certain room x, the three panoramas are located at respective taking positions L1, L2, and L3 in the three-dimensional model, and the location results are represented by corresponding three black dots. When a user views the three-dimensional model, the user wants to view a panorama photographed at a certain viewing angle in the room x based on the own requirement, and at this time, the user is assumed to click a mark point corresponding to the photographing position L1 in fig. 7 and enter the corresponding viewing angle to view the room x. Then, the corresponding panorama is displayed in the shooting orientation corresponding to the marker point, for example, the panorama is rotated and displayed in the shooting orientation.
The implementation process of the three-dimensional reconstruction of the house is introduced above. The implementation idea of house three-dimensional reconstruction can also be applied to application scenes of other similar three-dimensional model construction, such as the construction of a three-dimensional map.
For example, in practical applications, when a user wants to view a street view of an area (e.g., near a tourist attraction, near a residential district, near an office building), the map application may present a three-dimensional street view map to the user. The construction of the three-dimensional street view map can be implemented by means of the thought of three-dimensional reconstruction of the house.
Specifically, an area in which a three-dimensional street view map needs to be constructed is referred to as a target street block, and a two-dimensional map corresponding to the target street block, which corresponds to the house type map in the above description, can be obtained in advance, and a reference orientation angle is marked on the two-dimensional map. And shooting a panoramic picture of the target block to obtain one or more panoramic pictures, wherein the shooting positions and the shooting orientations of different panoramic pictures are different. Then, referring to the scheme described above, in conjunction with the two-dimensional map with the reference orientation angle marks, the shooting position and shooting orientation corresponding to each panorama are determined, where the shooting position of a certain panorama refers to the corresponding coordinate of the position in the two-dimensional map when the certain panorama is shot standing at a certain position, and the shooting orientation of a certain panorama refers to which reference orientation angle marked in the two-dimensional map the initial orientation of the camera matches when the certain panorama is shot. After the shooting position and the shooting orientation of the panorama are obtained, the construction of the three-dimensional street view map of the target street can be completed by referring to the scheme described above.
The house three-dimensional reconstruction apparatus according to one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these means can each be constructed using commercially available hardware components and by performing the steps taught in this disclosure.
Fig. 8 is a schematic structural diagram of a house three-dimensional reconstruction apparatus according to an embodiment of the present invention, and as shown in fig. 8, the house three-dimensional reconstruction apparatus includes: the system comprises an acquisition module 11, a pose determination module 12 and a reconstruction module 13.
The acquiring module 11 is configured to acquire a first panorama and a house type map corresponding to a room.
The pose determining module 12 is configured to determine a shooting orientation of the first panorama according to preset orientation information corresponding to the house type map; and determining the shooting position of the first panoramic image according to the shooting orientation of the first panoramic image and the house type image.
And the reconstruction module 13 is configured to perform three-dimensional reconstruction on the room according to the shooting orientation and the shooting position of the first panorama.
Optionally, the obtaining module 11 may be specifically configured to: acquiring an integral house type diagram of the house; analyzing a house type graph corresponding to the room from the integral house type graph according to the room identification in the integral house type graph; and determining that the first panoramic image corresponds to the room according to the room identifier corresponding to the first panoramic image.
Optionally, the pose determination module 12 may be specifically configured to: determining a bottom view corresponding to the first panorama; and determining the shooting orientation of the first panoramic image according to the structural features and preset orientation information presented in the user-type image and the structural features presented in the bottom image.
Optionally, the pose determination module 12 may be specifically configured to: determining a first structural drawing of the room from the first panorama; carrying out direction adjustment on the first structural diagram according to the shooting orientation of the first panoramic image; carrying out corner matching processing on the first structural drawing and the house-type drawing after the direction adjustment; and determining a mapping coordinate corresponding to a shooting reference point in the first structural drawing in the indoor type drawing according to the corner matching result, wherein the mapping coordinate is used as a shooting position of the first panoramic drawing.
Optionally, the obtaining module 11 may further be configured to: acquiring a plurality of panoramic pictures corresponding to the room; for any panoramic image in the plurality of panoramic images, determining the matching degree of the feature points between the any panoramic image and the rest panoramic images; and determining the first panoramic image with the characteristic point matching degree meeting the set requirement according to the characteristic point matching degree corresponding to each of the plurality of panoramic images.
Optionally, the pose determination module 12 may be further configured to: determining a shooting orientation and a shooting position of a second panorama of the plurality of panoramas, the second panorama being any panorama of the plurality of panoramas except the first panorama.
The device further comprises: the display module is used for displaying a mark point corresponding to the shooting position of the second panoramic image in the three-dimensional model obtained through the three-dimensional reconstruction; and responding to the selection operation of the mark point, and displaying the second panoramic image according to the shooting orientation of the second panoramic image.
Optionally, the pose determination module 12 may be specifically configured to: performing feature point matching on the second panoramic image and the first panoramic image so as to determine a rotation angle according to the matched feature points; determining the shooting orientation of the second panoramic image according to the shooting orientation of the first panoramic image and the rotation angle; determining a second structure diagram of the room according to the second panoramic image; and determining the shooting position of the second panoramic picture according to the second structure picture, the shooting direction of the second panoramic picture and the house type picture.
The three-dimensional house reconstruction apparatus shown in fig. 8 may perform the method provided in the embodiments shown in fig. 1 to fig. 7, and parts not described in detail in this embodiment may refer to the related description of the embodiments, which is not described herein again.
In one possible design, the structure of the three-dimensional house reconstruction apparatus shown in fig. 8 can be implemented as an electronic device. As shown in fig. 9, the electronic device may include: a processor 21 and a memory 22. Wherein the memory 22 has stored thereon executable code which, when executed by the processor 21, at least makes the processor 21 capable of implementing the house three-dimensional reconstruction method as provided in the embodiments illustrated in the foregoing fig. 1 to 7.
The electronic device may further include a communication interface 23 for communicating with other devices or a communication network.
In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the house three-dimensional reconstruction method provided in the foregoing embodiments shown in fig. 1 to 7.
The above-described apparatus embodiments are merely illustrative, wherein the various modules illustrated as separate components may or may not be physically separate. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course, can also be implemented by a combination of hardware and software. With this understanding in mind, the above-described aspects and portions of the present technology which contribute substantially or in part to the prior art may be embodied in the form of a computer program product, which may be embodied on one or more computer-usable storage media having computer-usable program code embodied therein, including without limitation disk storage, CD-ROM, optical storage, and the like.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A house three-dimensional reconstruction method is characterized by comprising the following steps:
acquiring a first panoramic view and a house type view corresponding to a room;
determining the shooting orientation of the first panoramic image according to preset orientation information corresponding to the house type image;
determining the shooting position of the first panoramic image according to the shooting orientation of the first panoramic image and the house type image;
and performing three-dimensional reconstruction on the room according to the shooting orientation and the shooting position of the first panoramic image.
2. The method of claim 1, wherein said determining the capture location of the first panorama from the capture orientation of the first panorama and the layout of the house comprises:
determining a first structural drawing of the room from the first panorama;
carrying out direction adjustment on the first structural diagram according to the shooting orientation of the first panoramic image;
carrying out corner matching processing on the first structural drawing and the house-type drawing after the direction adjustment;
and determining a mapping coordinate corresponding to a shooting reference point in the first structural drawing in the indoor type drawing according to the corner matching result, wherein the mapping coordinate is used as a shooting position of the first panoramic drawing.
3. The method of claim 1, wherein the obtaining a first panorama corresponding to a room comprises:
acquiring a plurality of panoramic pictures corresponding to the room;
for any panoramic image in the plurality of panoramic images, determining the matching degree of the feature points between the any panoramic image and the rest panoramic images;
and determining the first panoramic image with the characteristic point matching degree meeting the set requirement according to the characteristic point matching degree corresponding to each of the plurality of panoramic images.
4. The method of claim 1, further comprising:
acquiring a plurality of panoramic pictures corresponding to the room, wherein one panoramic picture in the plurality of panoramic pictures is used as the first panoramic picture;
determining a shooting orientation and a shooting position of a second panorama of the plurality of panoramas, the second panorama being any panorama of the plurality of panoramas except the first panorama;
displaying a mark point corresponding to the shooting position of the second panoramic image in the three-dimensional model obtained through the three-dimensional reconstruction;
and responding to the selection operation of the mark point, and displaying the second panoramic image according to the shooting orientation of the second panoramic image.
5. The method of claim 4, wherein determining the orientation in which a second panorama of the plurality of panoramas was photographed comprises:
performing feature point matching on the second panoramic image and the first panoramic image so as to determine a rotation angle according to the matched feature points;
and determining the shooting orientation of the second panoramic image according to the shooting orientation of the first panoramic image and the rotation angle.
6. The method of claim 4, wherein determining the capture location of the second of the plurality of panoramas comprises:
determining a second structure diagram of the room according to the second panoramic image;
and determining the shooting position of the second panoramic picture according to the second structure picture, the shooting direction of the second panoramic picture and the house type picture.
7. A three-dimensional reconstruction apparatus for a house, comprising:
the acquisition module is used for acquiring a first panoramic image and a house type image corresponding to a room;
the pose determining module is used for determining the shooting direction of the first panoramic image according to preset direction information corresponding to the house type image; determining the shooting position of the first panoramic image according to the shooting orientation of the first panoramic image and the house type image;
and the reconstruction module is used for performing three-dimensional reconstruction on the room according to the shooting direction and the shooting position of the first panoramic image.
8. The apparatus of claim 7, wherein the obtaining module is specifically configured to: acquiring a plurality of panoramic pictures corresponding to the room, wherein one panoramic picture in the plurality of panoramic pictures is used as the first panoramic picture;
the pose determination module is further to: determining a shooting orientation and a shooting position of a second panorama of the plurality of panoramas, the second panorama being any panorama of the plurality of panoramas except the first panorama;
the device further comprises: the display module is used for displaying a mark point corresponding to the shooting position of the second panoramic image in the three-dimensional model obtained through the three-dimensional reconstruction; and responding to the selection operation of the mark point, and displaying the second panoramic image according to the shooting orientation of the second panoramic image.
9. An electronic device, comprising: a memory, a processor; wherein the memory has stored thereon executable code which, when executed by the processor, causes the processor to perform the method of three-dimensional reconstruction of a house of any of claims 1 to 6.
10. A non-transitory machine-readable storage medium having stored thereon executable code, which when executed by a processor of an electronic device, causes the processor to perform the house three-dimensional reconstruction method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110327798.0A CN113298928A (en) | 2021-03-26 | 2021-03-26 | House three-dimensional reconstruction method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110327798.0A CN113298928A (en) | 2021-03-26 | 2021-03-26 | House three-dimensional reconstruction method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113298928A true CN113298928A (en) | 2021-08-24 |
Family
ID=77319146
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110327798.0A Pending CN113298928A (en) | 2021-03-26 | 2021-03-26 | House three-dimensional reconstruction method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113298928A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113920270A (en) * | 2021-12-15 | 2022-01-11 | 深圳市其域创新科技有限公司 | Layout reconstruction method and system based on multi-view panorama |
| CN114554092A (en) * | 2022-02-24 | 2022-05-27 | 北京有竹居网络技术有限公司 | Equipment control method and device and electronic equipment |
| CN114663618A (en) * | 2022-03-03 | 2022-06-24 | 北京城市网邻信息技术有限公司 | Three-dimensional reconstruction and correction method, device, equipment and storage medium |
| CN115830162A (en) * | 2022-11-21 | 2023-03-21 | 北京城市网邻信息技术有限公司 | Home map display method and device, electronic equipment and storage medium |
-
2021
- 2021-03-26 CN CN202110327798.0A patent/CN113298928A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113920270A (en) * | 2021-12-15 | 2022-01-11 | 深圳市其域创新科技有限公司 | Layout reconstruction method and system based on multi-view panorama |
| CN114554092A (en) * | 2022-02-24 | 2022-05-27 | 北京有竹居网络技术有限公司 | Equipment control method and device and electronic equipment |
| CN114663618A (en) * | 2022-03-03 | 2022-06-24 | 北京城市网邻信息技术有限公司 | Three-dimensional reconstruction and correction method, device, equipment and storage medium |
| CN114663618B (en) * | 2022-03-03 | 2022-11-29 | 北京城市网邻信息技术有限公司 | Three-dimensional reconstruction and correction method, device, equipment and storage medium |
| CN115830162A (en) * | 2022-11-21 | 2023-03-21 | 北京城市网邻信息技术有限公司 | Home map display method and device, electronic equipment and storage medium |
| CN115830162B (en) * | 2022-11-21 | 2023-11-14 | 北京城市网邻信息技术有限公司 | House type diagram display method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10176589B2 (en) | Method and system for completing point clouds using planar segments | |
| JP6951595B2 (en) | Housing data collection and model generation methods | |
| CN113298928A (en) | House three-dimensional reconstruction method, device, equipment and storage medium | |
| JP5740884B2 (en) | AR navigation for repeated shooting and system, method and program for difference extraction | |
| US10026218B1 (en) | Modeling indoor scenes based on digital images | |
| JP5093053B2 (en) | Electronic camera | |
| CN110874818B (en) | Image processing and virtual space construction method, device, system and storage medium | |
| JP6180647B2 (en) | Indoor map construction apparatus and method using cloud points | |
| JP2005135355A (en) | Data authoring processing apparatus | |
| CN111260793B (en) | Remote virtual-real high-precision matching positioning method for augmented and mixed reality | |
| WO2022088881A1 (en) | Method, apparatus and system for generating a three-dimensional model of a scene | |
| CN114640833A (en) | Projection picture adjusting method and device, electronic equipment and storage medium | |
| JP7241812B2 (en) | Information visualization system, information visualization method, and program | |
| JP2000050156A (en) | News supporting system | |
| Nyland et al. | Capturing, processing, and rendering real-world scenes | |
| CN110191284B (en) | Method and device for collecting data of house, electronic equipment and storage medium | |
| CN114089836B (en) | Labeling method, terminal, server and storage medium | |
| EP4012654A2 (en) | Feature determination, measurement, and virtualization from 2-d image capture | |
| JP2008203991A (en) | Image processor | |
| CN113822936A (en) | Data processing method and device, computer equipment and storage medium | |
| CN113763561B (en) | POI data generation method and device, storage medium and electronic equipment | |
| US20240087157A1 (en) | Image processing method, recording medium, image processing apparatus, and image processing system | |
| Lichtenauer et al. | A semi-automatic procedure for texturing of laser scanning point clouds with google streetview images | |
| Mayer et al. | Comparison of photogrammetric and computer vision techniques-3d reconstruction and visualization of wartburg castle | |
| CN114442895A (en) | Three-dimensional model construction method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240311 Address after: # 03-06, Lai Zan Da Building 1, 51 Belarusian Road, Singapore Applicant after: Alibaba Innovation Co. Country or region after: Singapore Address before: Room 01, 45th Floor, AXA Building, 8 Shanton Road, Singapore Applicant before: Alibaba Singapore Holdings Ltd. Country or region before: Singapore |