CN111383355A - Three-dimensional point cloud completion method and device and computer readable storage medium - Google Patents

Three-dimensional point cloud completion method and device and computer readable storage medium Download PDF

Info

Publication number
CN111383355A
CN111383355A CN202010257587.XA CN202010257587A CN111383355A CN 111383355 A CN111383355 A CN 111383355A CN 202010257587 A CN202010257587 A CN 202010257587A CN 111383355 A CN111383355 A CN 111383355A
Authority
CN
China
Prior art keywords
point
point cloud
compensated
supplemented
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010257587.XA
Other languages
Chinese (zh)
Other versions
CN111383355B (en
Inventor
田明哲
谢哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
You Can See Beijing Technology Co ltd AS
Original Assignee
Beike Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beike Technology Co Ltd filed Critical Beike Technology Co Ltd
Priority to CN202010257587.XA priority Critical patent/CN111383355B/en
Publication of CN111383355A publication Critical patent/CN111383355A/en
Application granted granted Critical
Publication of CN111383355B publication Critical patent/CN111383355B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Architecture (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The embodiment of the disclosure discloses a three-dimensional point cloud completion method, a device and a computer readable storage medium, wherein the method comprises the following steps: acquiring a three-dimensional point cloud corresponding to a target room, wherein the three-dimensional point cloud comprises a top surface point cloud, a ground point cloud and a placed article point cloud; generating a top surface point cloud projection image based on the top surface point cloud, and generating a ground point cloud projection image based on the ground point cloud and the put article point cloud; determining at least one point cloud area to be complemented from the top surface point cloud projection image and the ground point cloud projection image; and for each point cloud area to be compensated in at least one point cloud area to be compensated, based on the position of the point cloud area to be compensated, utilizing the point clouds around the point cloud area to be compensated to compensate the point cloud area to be compensated. The embodiment of the disclosure can improve the accuracy of completing the cavity in the three-dimensional point cloud, and increase the integrity and the aesthetic degree of the house model.

Description

Three-dimensional point cloud completion method and device and computer readable storage medium
Technical Field
The disclosure relates to the technical field of computers, in particular to a three-dimensional point cloud completion method and device and a computer readable storage medium.
Background
In some cases, one often needs to have a rough picture of the situation of the whole house in advance. In general, the preliminary understanding can be performed through house information description, three-dimensional scene roaming, three-dimensional models, house type diagrams, and the like. Therefore, the accuracy, the integrity and the attractiveness of the model are factors influencing the intuitive understanding of the house model.
Disclosure of Invention
The embodiment of the disclosure provides a three-dimensional point cloud completion method and device, a computer-readable storage medium and three-dimensional point cloud completion equipment.
The embodiment of the disclosure provides a three-dimensional point cloud completion method, which comprises the following steps: acquiring a three-dimensional point cloud corresponding to a target room, wherein the three-dimensional point cloud comprises a top surface point cloud, a ground point cloud and a placed article point cloud; generating a top surface point cloud projection image based on the top surface point cloud, and generating a ground point cloud projection image based on the ground point cloud and the put article point cloud; determining at least one point cloud area to be complemented from the top surface point cloud projection image and the ground point cloud projection image; and for each point cloud area to be compensated in at least one point cloud area to be compensated, based on the position of the point cloud area to be compensated, utilizing the point clouds around the point cloud area to be compensated to compensate the point cloud area to be compensated.
In some embodiments, generating a top point cloud projection image based on the top point cloud and generating a ground point cloud projection image based on the ground point cloud and the posed item point cloud comprises: based on coordinates of points included in the top surface point cloud, mapping the top surface point cloud to a first image with a preset size and a preset color to obtain a top surface point cloud projection image; and mapping the ground point cloud and other point clouds onto a second image with a preset size and a preset color based on the coordinates of the ground point cloud and other point clouds to obtain a ground point cloud projection image.
In some embodiments, complementing the point cloud area to be complemented with the point clouds around the point cloud area to be complemented based on the position of the point cloud area to be complemented includes: supplementing a point set to be supplemented, which consists of a plurality of points to be supplemented, into the point cloud area to be supplemented based on a preset point interval; determining the height of each point to be supplemented in the point set to be supplemented by utilizing the three-dimensional point cloud near the point set to be supplemented; and completing the three-dimensional cavity corresponding to the point cloud area to be completed based on the height of each point to be completed.
In some embodiments, determining the height of each point to be compensated in the set of points to be compensated using the three-dimensional point cloud in the vicinity of the set of points to be compensated comprises: in response to determining that the point cloud area to be compensated is located on the top surface, for each point to be compensated in the point cloud area to be compensated, determining a point which is closest to the point to be compensated in the points included in the top surface point cloud; and determining the height of the determined point as the height of the point to be compensated.
In some embodiments, determining the height of each point to be compensated in the set of points to be compensated using the three-dimensional point cloud in the vicinity of the set of points to be compensated comprises: in response to the fact that the point cloud area to be supplemented is located on the ground, determining whether a vertical plane exists in a preset range of the point to be supplemented or not for each point to be supplemented in a point set to be supplemented included in the point cloud area to be supplemented based on the three-dimensional point cloud; in response to determining that a vertical plane exists, dividing the vertical plane to obtain a plurality of blocks; and projecting the point to be compensated to a vertical plane, and determining the height of the block where the projected point is located as the height of the point to be compensated.
In some embodiments, determining the height of each point to be compensated in the set of points to be compensated using the three-dimensional point cloud in the vicinity of the set of points to be compensated further comprises: in response to the fact that the vertical plane does not exist in the preset range of the point to be compensated, dividing a point set to be compensated, which is included in the point cloud area to be compensated, into at least one point subset to be compensated; determining a peripheral point subset to be supplemented from at least one point subset to be supplemented; for each point to be supplemented in the peripheral point to be supplemented subset, determining a point closest to the point to be supplemented from the three-dimensional point cloud, and determining the height of the determined point as the height of the point to be supplemented; determining the peripheral point subset to be compensated after the height is determined as a reference point set; re-determining a peripheral point subset to be supplemented from at least one point subset to be supplemented, and executing the following supplementing steps based on the re-determined peripheral point subset to be supplemented: for each point to be compensated in the peripheral point to be compensated subset, determining the distance weight between the point to be compensated and each point in the reference point set, and performing weighted summation on the heights of the points in the reference point set by using the distance weight to obtain the height of the point to be compensated; adding the point to be compensated as a reference point into a reference point set; and in response to the fact that at least one to-be-compensated point subset is determined to have a to-be-compensated point subset which is not subjected to compensation, re-determining a peripheral to-be-compensated point subset from the to-be-compensated point subset which is not subjected to compensation, and continuing to execute the compensation step by using the re-determined peripheral to-be-compensated point subset.
In some embodiments, dividing the set of points to be compensated included in the point cloud area to be compensated into at least one subset of points to be compensated includes: the following partitioning steps are performed: determining the number of points to be compensated in a circle which takes the points to be compensated as the center of the circle and takes the point distance as the radius for each point to be compensated in the point cloud area to be compensated; determining the points to be compensated corresponding to the circle centers of circles of which the number of the points to be compensated is less than the preset number as peripheral points to be compensated; determining each determined peripheral point to be supplemented as a subset of the points to be supplemented and removing the points from the set of the points to be supplemented; determining whether the number of the points to be compensated which are remained after removing the subset of the points to be compensated is zero or not; if the number is zero, ending the dividing step; and if the number is not zero, taking the residual points to be compensated after the point subset to be compensated is removed as a new point set to be compensated, and continuing to execute the dividing step.
In some embodiments, determining at least one point cloud area to be complemented from the top point cloud projection image and the ground point cloud projection image comprises: determining at least one candidate point cloud area to be complemented from the top surface point cloud projection image and the ground point cloud projection image; and filtering candidate point cloud areas to be compensated which do not accord with preset conditions in the at least one candidate point cloud area to be compensated to obtain the at least one point cloud area to be compensated.
In some embodiments, the preset condition includes at least one of: the length-width ratio of a circumscribed rectangle of the candidate region to be compensated is within a preset proportion range; the area of the candidate region to be compensated is within a preset area range; the area ratio of the candidate to-be-compensated area to the circumscribed rectangle is larger than or equal to a preset ratio.
According to another aspect of the embodiments of the present disclosure, there is provided a three-dimensional point cloud complementing apparatus, including: the system comprises an acquisition module, a storage module and a display module, wherein the acquisition module is used for acquiring three-dimensional point clouds corresponding to a target room, and the three-dimensional point clouds comprise a top surface point cloud, a ground point cloud and a put article point cloud; the generation module is used for generating a top surface point cloud projection image based on the top surface point cloud and generating a ground point cloud projection image based on the ground point cloud and the put article point cloud; the determining module is used for determining at least one point cloud area to be supplemented from the top surface point cloud projection image and the ground point cloud projection image; and the complementing module is used for complementing each point cloud area to be complemented in at least one point cloud area to be complemented by utilizing the point clouds around the point cloud area to be complemented based on the position of the point cloud area to be complemented.
According to another aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing a computer program for executing the above three-dimensional point cloud complementing method.
According to another aspect of the embodiments of the present disclosure, there is provided a three-dimensional point cloud complementing apparatus including: at least one processor; a memory coupled to the at least one processor; the memory stores instructions capable of being executed by the at least one processor, and the at least one processor realizes the three-dimensional point cloud completion method by executing the instructions stored by the memory.
Based on the three-dimensional point cloud complementing method, the three-dimensional point cloud complementing device, the computer-readable storage medium and the three-dimensional point cloud complementing equipment provided by the embodiments of the disclosure, the top surface point cloud projection image and the ground surface point cloud projection image are generated, at least one point cloud area to be complemented is determined from the top surface point cloud projection image and the ground surface point cloud projection image, and finally the point cloud area to be complemented is complemented by using the point clouds around the point cloud area to be complemented based on the position of the point cloud area to be complemented, so that the position characteristics of the point cloud area to be complemented are effectively utilized, the point cloud area to be complemented is complemented, the accuracy of hole complementing in the three-dimensional point cloud is improved, and the integrity and the aesthetic degree of a house model are increased.
The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing in more detail embodiments of the present disclosure with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure and not to limit the disclosure. In the drawings, like reference numbers generally represent like parts or steps.
Fig. 1 is a system diagram to which the present disclosure is applicable.
Fig. 2 is a schematic flow chart of a three-dimensional point cloud completion method according to an exemplary embodiment of the present disclosure.
Fig. 3 is an exemplary schematic diagram of a provided top point cloud projection image and ground point cloud projection image of an embodiment of the present disclosure.
Fig. 4 is a schematic flow chart of a three-dimensional point cloud completion method according to another exemplary embodiment of the present disclosure.
Fig. 5 is a schematic flow chart of a three-dimensional point cloud completion method according to another exemplary embodiment of the present disclosure.
Fig. 6 is an exemplary schematic diagram of dividing a vertical plane provided by an embodiment of the present disclosure.
Fig. 7 is a schematic flowchart of a three-dimensional point cloud completion method according to another exemplary embodiment of the present disclosure.
Fig. 8 is an exemplary diagram of a subset of points to be complemented provided by an embodiment of the disclosure.
Fig. 9 is a schematic flowchart of a three-dimensional point cloud completion method according to another exemplary embodiment of the present disclosure.
Fig. 10 is a schematic structural diagram of a three-dimensional point cloud completion apparatus according to an exemplary embodiment of the present disclosure.
Fig. 11 is a schematic structural diagram of a three-dimensional point cloud completion apparatus according to another exemplary embodiment of the present disclosure.
Fig. 12 is a structural diagram of a three-dimensional point cloud complementing device according to an exemplary embodiment of the disclosure.
Detailed Description
Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of the embodiments of the present disclosure and not all embodiments of the present disclosure, with the understanding that the present disclosure is not limited to the example embodiments described herein.
It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present disclosure are used merely to distinguish one element from another, and are not intended to imply any particular technical meaning, nor is the necessary logical order between them.
It is also understood that in embodiments of the present disclosure, "a plurality" may refer to two or more and "at least one" may refer to one, two or more.
It is also to be understood that any reference to any component, data, or structure in the embodiments of the disclosure, may be generally understood as one or more, unless explicitly defined otherwise or stated otherwise.
In addition, the term "and/or" in the present disclosure is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the former and latter associated objects are in an "or" relationship.
It should also be understood that the description of the various embodiments of the present disclosure emphasizes the differences between the various embodiments, and the same or similar parts may be referred to each other, so that the descriptions thereof are omitted for brevity.
Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.
Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
The disclosed embodiments may be applied to electronic devices such as terminal devices, computer systems, servers, etc., which are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known terminal devices, computing systems, environments, and/or configurations that may be suitable for use with electronic devices, such as terminal devices, computer systems, servers, and the like, include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set top boxes, programmable consumer electronics, network pcs, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above systems, and the like.
Electronic devices such as terminal devices, computer systems, servers, etc. may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.
Summary of the application
In the process of acquiring the three-dimensional point cloud, the scanned object often faces data loss caused by reasons of shading by other objects or overlarge illumination intensity, and the scanned object visually appears as a hollow area. Therefore, in order to make the house model more complete and beautiful, it is necessary to complement these hollow areas.
Exemplary System
Fig. 1 illustrates an exemplary system architecture 100 to which a three-dimensional point cloud completion method or a three-dimensional point cloud completion apparatus of an embodiment of the present disclosure may be applied.
As shown in fig. 1, system architecture 100 may include terminal device 101, network 102, and server 103. Network 102 is the medium used to provide communication links between terminal devices 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
A user may use terminal device 101 to interact with server 103 over network 102 to receive or send messages and the like. Various communication client applications, such as a three-dimensional model application, a house design application, a web browser application, and the like, may be installed on the terminal device 101.
The terminal device 101 may be various electronic devices including, but not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle-mounted terminal (e.g., a car navigation terminal), etc., and a fixed terminal such as a digital TV, a desktop computer, etc.
The server 103 may be a server that provides various services, such as a background point cloud processing server that processes a three-dimensional point cloud uploaded by the terminal device 101. The background point cloud processing server can process the received three-dimensional point cloud to obtain the three-dimensional point cloud after the cavity area is completed.
It should be noted that the three-dimensional point cloud complementing method provided by the embodiment of the present disclosure may be executed by the server 103, or may be executed by the terminal device 101, and accordingly, the three-dimensional point cloud complementing device may be disposed in the server 103, or may be disposed in the terminal device 101.
It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation. In the case that the three-dimensional point cloud does not need to be acquired from a remote place, the system architecture may not include a network, and only include a server or a terminal device.
Exemplary method
Fig. 2 is a schematic flow chart of a three-dimensional point cloud completion method according to an exemplary embodiment of the present disclosure. The embodiment can be applied to an electronic device (such as the terminal device 101 or the server 103 shown in fig. 1), and as shown in fig. 2, the method includes the following steps:
step 201, obtaining a three-dimensional point cloud corresponding to a target room.
In this embodiment, the electronic device may obtain the three-dimensional point cloud corresponding to the target room from a remote location or from a local location. The target room may be a room scanned three-dimensionally by a three-dimensional point cloud acquisition device (e.g., a laser radar, a stereo camera, etc.). The three-dimensional point cloud collection device can collect the three-dimensional point cloud, and a three-dimensional model of a target room can be constructed by using the three-dimensional point cloud. The three-dimensional point cloud comprises a top surface point cloud, a ground surface point cloud and a placed article point cloud. The top surface point cloud is used for representing a ceiling of a target room, the ground surface point cloud is used for representing a ground of the target room, and the put article point cloud is used for representing various articles placed in the target room. In general, in the model of the target room, the point cloud located on the top surface may be used as the top surface point cloud, the point cloud located on the ground may be used as the ground surface point cloud, and the point cloud located inside the model space may be used as the item placement point cloud. It should be noted that the three-dimensional point cloud may also include a point cloud of a vertical plane, which is used to represent a plane perpendicular to the top surface or the ground, such as a wall surface, a vertical plane on which articles are placed, and the like.
In general, the Y axis of the coordinate system is oriented downward, the XOZ plane is a horizontal plane, and the normal vector of each plane is (nx, ny, nz). Ny for the top surface>Tceil(ii) a For the ground, ny<Tfloor(ii) a For the vertical plane, nx2+nz2>Twall，TceilCan be 0.985, TfloorCan be-0.985, TwallCan be 0.985.
Step 202, generating a top surface point cloud projection image based on the top surface point cloud, and generating a ground point cloud projection image based on the ground point cloud and the put article point cloud.
In this embodiment, the electronic device may generate a top point cloud projection image based on the top point cloud and a ground point cloud projection image based on the ground point cloud and the posed article point cloud. As an example, the electronic device may project an area covered by the top point cloud onto a horizontal plane where the top surface of the model of the target room is located, and then map the projected area on the horizontal plane onto a first image with a preset size according to a preset ratio. The electronic equipment can also project the area covered by the ground point cloud and the area covered by the article placing point cloud onto a horizontal plane where the ground of the model of the target room is located, and then the projected area on the horizontal plane is mapped onto a second image with a preset size according to a preset proportion.
In some alternative implementations, step 202 may be performed as follows:
firstly, based on coordinates of points included in the top surface point cloud, the top surface point cloud is mapped to a first image with a preset size and a preset color, and a top surface point cloud projection image is obtained.
And then, mapping the ground point cloud and other point clouds onto a second image with a preset size and a preset color based on the coordinates of the ground point cloud and other point clouds to obtain a ground point cloud projection image.
Specifically, as an example, the electronic device may initialize first and second images, where the first and second images are of Size and the pixels are all 0 (i.e., black). Assuming that the widths of the point cloud in the x-axis direction and the z-axis direction are Wx and Wz (unit: M), respectively, the Size is max (M × Wx +200, M × Wz +200), and the unit is pixel. Where M is the mapping coefficient from the point cloud to the image and 200 is the margin left around the image. Thereafter, the point cloud (px, pz) is mapped to the pixel (r, c) according to equation (1), and a top point cloud projection image (as shown in fig. 3 (a)) and a ground point cloud projection image (as shown in fig. 3 (b)) are obtained.
Figure BDA0002437997850000081
Where 100 is the margin left around the image, minX and minZ are the minimum of the x and z coordinates of the point cloud, M is the mapping coefficient from the point cloud to the image, where 50 is taken, and function floor () is a floor function, where the values in brackets are rounded because the point cloud coordinates contain decimal numbers and the pixel coordinates are integers.
According to the optional implementation mode, the three-dimensional point cloud is mapped to the first image and the second image which are preset in size and color, so that the standardized top surface point cloud projection image and the standardized ground surface point cloud projection image can be obtained, and the point cloud area to be supplemented can be accurately obtained by comparing the top surface point cloud projection image and the ground surface point cloud projection image.
Step 203, determining at least one point cloud area to be compensated from the top surface point cloud projection image and the ground point cloud projection image.
In this embodiment, the electronic device may determine at least one point cloud area to be complemented from the top point cloud projection image and the ground point cloud projection image. Specifically, as an example, the electronic device may determine, from the top surface point cloud projection image and the ground point cloud projection image, an area to which the three-dimensional point cloud is not mapped as a point cloud area to be complemented.
In some alternative implementations, step 203 may be performed as follows:
firstly, at least one candidate point cloud area to be supplemented is determined from the top surface point cloud projection image and the ground point cloud projection image. Specifically, the electronic device may determine, from the top surface point cloud projection image and the ground point cloud projection image, all regions to which the three-dimensional point cloud is not mapped as candidate point cloud regions to be complemented.
And then, filtering candidate point cloud areas to be compensated which do not accord with preset conditions in the at least one candidate point cloud area to be compensated to obtain the at least one point cloud area to be compensated. Candidate point cloud regions to be supplemented which do not meet preset conditions are usually small in area or represent facilities such as walls and the like, and the regions do not usually need to be supplemented. According to the implementation mode, the candidate point cloud area to be supplemented which does not accord with the preset condition is filtered, so that the cavity area to be supplemented can be accurately obtained, and the accuracy of three-dimensional point cloud supplementation and the integrity of the model are improved.
In some optional implementations, the preset condition includes at least one of:
under the condition one, the aspect ratio of the circumscribed rectangle of the candidate region to be compensated is within the preset proportion range.
And secondly, the area of the candidate region to be compensated is within the preset area range.
And under the third condition, the area ratio of the area of the candidate to-be-compensated region to the area of the circumscribed rectangle is larger than or equal to the preset ratio.
By the preset conditions, the candidate to-be-supplemented areas in the slender shape, the candidate to-be-supplemented areas with smaller areas, the candidate to-be-supplemented areas with more dispersed coverage areas and the like can be filtered from the candidate to-be-supplemented areas, so that the accuracy of obtaining the to-be-supplemented areas can be improved, and a complete room model which accords with the actual situation can be obtained.
And 204, for each point cloud area to be compensated in at least one point cloud area to be compensated, based on the position of the point cloud area to be compensated, utilizing the point clouds around the point cloud area to be compensated to compensate the point cloud area to be compensated.
In this embodiment, for each point cloud area to be compensated in the at least one point cloud area to be compensated, the electronic device may utilize the point clouds around the point cloud area to be compensated to compensate the point cloud area to be compensated based on the position of the point cloud area to be compensated. Specifically, for a certain point cloud area to be supplemented, the position of the point cloud area to be supplemented may be located on the top surface of the model of the target room, the ground surface, or the surface on which the object is placed, and since the horizontal plane coordinates of the point cloud area to be supplemented are known (e.g., x coordinates and z coordinates), the electronic device may assign the height (e.g., y coordinates) of the three-dimensional point cloud near the point cloud area to be supplemented to the height of each point in the point cloud area to be supplemented, or perform fitting by using the height of the three-dimensional point cloud near the point cloud area to be supplemented, to obtain the height of each point in the point cloud area to be supplemented.
According to the method provided by the embodiment of the disclosure, the top point cloud projection image and the ground point cloud projection image are generated, at least one point cloud area to be supplemented is determined from the top point cloud projection image and the ground point cloud projection image, and finally the point cloud area to be supplemented is supplemented by utilizing the point clouds around the point cloud area to be supplemented based on the position of the point cloud area to be supplemented, so that the position characteristics of the point cloud area to be supplemented are effectively utilized to supplement the point cloud area to be supplemented, the accuracy of supplementing the cavity in the three-dimensional point cloud is improved, and the integrity and the aesthetic degree of the house model are increased.
With further reference to fig. 4, a schematic flow chart of yet another embodiment of the three-dimensional point cloud completion method is shown. As shown in fig. 4, based on the embodiment shown in fig. 2, for each point cloud area to be compensated in at least one point cloud area to be compensated, the following steps may be performed:
step 2041, supplementing a point set to be supplemented, which is composed of a plurality of points to be supplemented, into the point cloud region to be supplemented based on a preset point interval.
Specifically, the electronic device may uniformly distribute a plurality of points to be compensated into the cloud area of the points to be compensated according to the point spacing, and since the horizontal plane coordinates of the cloud area of the points to be compensated are known, the horizontal plane coordinates of each point to be compensated in the set of points to be compensated may be determined.
Step 2042, determining the height of each point to be compensated in the point set to be compensated by using the three-dimensional point cloud near the point set to be compensated.
Specifically, each point cloud area to be supplemented may be located at different positions in the model of the target room, so that the point set to be supplemented may be analyzed in different ways according to the position of the point cloud area to be supplemented, and the height of each point to be supplemented is obtained.
And 2043, completing the three-dimensional cavity corresponding to the point cloud area to be completed based on the height of each point to be completed.
Specifically, after the horizontal plane coordinates and the height of each point to be compensated in the point set to be compensated are determined, the three-dimensional cavity corresponding to the point cloud area to be compensated can be compensated in the three-dimensional point cloud according to the coordinates of each point to be compensated.
In the method provided by the embodiment corresponding to fig. 4, the point cloud area to be supplemented is filled with the point set to be supplemented, then the three-dimensional point cloud near the point set to be supplemented is utilized to determine the height of each point to be supplemented in the point set to be supplemented, and then the three-dimensional cavity corresponding to the point cloud area to be supplemented is supplemented, so that the three-dimensional point cloud near the point to be supplemented is effectively utilized to determine the height of the point to be supplemented, that is, the coordinates of each point to be supplemented are accurately determined, and the efficiency and accuracy of three-dimensional point cloud supplementation are improved.
In some optional implementations, the step 2042 may include the following steps:
in response to determining that the point cloud area to be compensated is located on the top surface, for each point to be compensated in the point cloud area to be compensated, the electronic device may perform the following steps:
firstly, the point which is closest to the point to be compensated in the points included in the top surface point cloud is determined.
Then, the height of the determined point is determined as the height of the point to be compensated.
In this implementation, the following a priori knowledge is utilized: because the top surface point cloud is located on the top surface of the room, the heights of all points included in the top surface point cloud are the same, and therefore, the height of each point to be completed is only required to be set to be the height of the surrounding points, and therefore the three-dimensional point cloud completion efficiency and accuracy can be improved.
In some alternative implementations, as shown in fig. 5, the step 2042 may include the following steps:
step 204201, in response to determining that the point cloud area to be compensated is located on the ground, determining whether a vertical plane exists in a preset range of the point to be compensated based on the three-dimensional point cloud for each point to be compensated in the point set to be compensated included in the point cloud area to be compensated.
The preset range may be determined in various manners, for example, the preset range may be a range included in a circle with the point to be completed as a circle and the preset length as a radius, or may be a range included in a rectangle with the point to be completed as a center and the preset length as a side length. The vertical plane can be obtained by using a point cloud representing the vertical plane, which is obtained by dividing the three-dimensional point cloud in advance.
In response to determining that a vertical plane exists, the vertical plane is divided, resulting in a plurality of partitions, step 204202.
As an example, as shown in FIG. 6, a vertical plane is shown, which is divided into a plurality of blocks, each block having a width w and a height yi(i＝1,2,3,…)。
And 204203, projecting the point to be compensated to the vertical plane, and determining the height of the block where the projected point is located as the height of the point to be compensated.
As shown in FIG. 6, the height of the block where the projection point of the point p1 to be compensated is located is y1The height of the point p1 to be compensated is set to y1Based on the same method, the heights of the points p2 and p3 to be compensated are set to y respectively7And y14。
In this implementation, the following a priori knowledge is utilized: if the point to be repaired is located at a position corresponding to an article such as a kitchen range, a table, etc., the height of the point to be repaired coincides with the height of the article. The implementation mode realizes that when a vertical plane exists near the point to be supplemented, the height of the vertical plane is directly endowed to the point to be supplemented, and the efficiency and the accuracy of three-dimensional point cloud supplementation are improved.
In some alternative implementations, as shown in fig. 7, after the step 204202, the electronic device may further perform the following steps:
step 204204, in response to determining that the vertical plane does not exist within the preset range of the point to be compensated, dividing the point to be compensated set included in the point cloud area to be compensated into at least one point subset to be compensated.
Specifically, the electronic device may sequentially determine the subset of points to be compensated in an order from the periphery of the set of points to be compensated to the inside. For example, the point cloud area to be compensated is divided into a plurality of closed-loop areas with the same width, and the point to be compensated covered by each closed-loop area is a subset of the point to be compensated.
At step 204205, a peripheral point subset to be complemented is determined from the at least one point subset to be complemented.
And the peripheral point subset to be compensated is a subset located at the outermost periphery of the point set to be compensated. As shown in fig. 8, the subset of points to be compensated, which is numbered 1, is a peripheral subset of points to be compensated.
And 204206, determining the point closest to the point to be compensated from the three-dimensional point cloud for each point to be compensated in the peripheral point to be compensated subset, and determining the height of the determined point as the height of the point to be compensated.
And step 204207, determining the peripheral point subset to be compensated after the height is determined as a reference point set.
After the step is executed, the reference point set is added after the height of the point to be compensated with the number of 1 shown in fig. 8 is determined.
At step 204208, a peripheral point-to-be-compensated subset is re-determined from the at least one point-to-be-compensated subset, and the compensation step is performed based on the re-determined peripheral point-to-be-compensated subset (i.e., step 204209).
Specifically, as shown in fig. 8, the peripheral point-to-be-compensated subset is newly determined as the point-to-be-compensated subset numbered 2.
204209, determining the distance weight between each point in the peripheral to-be-compensated point subset and each point in the reference point set, and performing weighted summation on the heights of the points in the reference point set by using the distance weights to obtain the height of each point to be compensated; and adding the point to be compensated into the reference point set as a reference point.
The distance weight is used for representing the influence degree of the distance between the point to be completed and each point in the reference point set on the height of the point to be completed. I.e. the closer the distance, the higher the degree of influence, and the further the distance, the lower the degree of influence.
As an example, for a certain point to be compensated, the height of the point to be compensated can be determined by using the following formula (2):
Figure BDA0002437997850000121
wherein HiDenotes the height of the point i to be complemented, m is the number of reference points in the set of reference points, j denotes the number of reference points, λjIs the distance weight of the point i to be compensated and the reference point j, YjIs the height of reference point j. Wherein λ isjThe dot spacing is divided by the distance between the point i to be compensated and the reference point j.
Step 204210, determine whether there is a point to be compensated subset that is not compensated in at least one point to be compensated subset.
Step 204211, in response to determining that there is a to-be-compensated point subset that is not compensated in at least one to-be-compensated point subset, re-determining a peripheral to-be-compensated point subset from the to-be-compensated point subset, and continuing to perform the compensation step using the re-determined peripheral to-be-compensated point subset.
Specifically, when the completion step is performed for the second time, the peripheral point-to-be-completed subset is the point-to-be-completed subset numbered 3 as shown in fig. 8, and the reference point set includes the point-to-be-completed subsets numbered 1 and 2 after the completion height.
It should be appreciated that the loop exits when there is no unfulfilled subset of points to be complemented in at least one subset of points to be complemented. By executing the steps 204204-204210, the height of the point to be supplemented is determined in a quasi-layer manner by using the three-dimensional point cloud around the point set to be supplemented when a vertical plane does not exist in the preset range, and the height of the point to be supplemented is determined in a manner of weighting the heights of the surrounding points when the height is determined, so that the accuracy of supplementing the point cloud area to be supplemented when the vertical plane does not exist around the point cloud area to be supplemented is further improved, and the completeness of the building model is further improved.
In some alternative implementations, as shown in fig. 9, the step 204 may be performed as follows:
and step A, determining the number of points to be compensated in a circle which takes the points to be compensated as the center of the circle and the point distance as the radius for each point to be compensated in the point cloud area to be compensated in the point set to be compensated.
It should be noted that, if the point to be compensated is located on the circle, it is determined that the point to be compensated is located in the circle.
And step B, determining the points to be compensated corresponding to the circle centers of the circles of which the number of the points to be compensated is less than the preset number as the peripheral points to be compensated.
Wherein the preset number is typically set to 4. As shown in fig. 8, each point to be compensated, which is numbered 1, is taken as a center (the center of a square in the drawing), a circle (such as a circle in the drawing) with a point interval as a radius is taken, the point to be compensated, which corresponds to the center of the circle, is removed, and the point to be compensated, which is located in the circle, is less than 4, so that each point to be compensated, which is numbered 1, is taken as a peripheral point to be compensated.
And step C, determining each determined peripheral point to be supplemented as a subset of the points to be supplemented and removing the points from the set of the points to be supplemented.
And D, determining whether the number of the points to be compensated which are remained after the point subset to be compensated is removed is zero.
And E, if the number is zero, ending the dividing step.
And F, if the number is not zero, taking the residual points to be compensated after the point subset to be compensated is removed as a new point set to be compensated, and continuing to execute the dividing step (steps A-D).
By circularly executing the steps A to D, the point set to be compensated can be sequentially divided into a plurality of point subsets to be compensated from outside to inside, so that the accuracy of determining the height of the points to be compensated from outside to inside is improved.
Exemplary devices
Fig. 10 is a schematic structural diagram of a three-dimensional point cloud completion apparatus according to an exemplary embodiment of the present disclosure. The embodiment can be applied to an electronic device, as shown in fig. 10, the three-dimensional point cloud complementing device includes: an obtaining module 1001, configured to obtain a three-dimensional point cloud corresponding to a target room, where the three-dimensional point cloud includes a top surface point cloud, a ground point cloud, and a placed article point cloud; a generating module 1002, configured to generate a top point cloud projection image based on the top point cloud, and generate a ground point cloud projection image based on the ground point cloud and the put item point cloud; a determining module 1003, configured to determine at least one point cloud area to be complemented from the top point cloud projection image and the ground point cloud projection image; a complementing module 1004, configured to, for each point cloud area to be complemented in the at least one point cloud area to be complemented, complement the point cloud area to be complemented by using the point clouds around the point cloud area to be complemented based on the position of the point cloud area to be complemented.
In this embodiment, the obtaining module 1001 may obtain the three-dimensional point cloud corresponding to the target room from a remote location or a local location. The target room may be a room scanned three-dimensionally by a three-dimensional point cloud acquisition device (e.g., a laser radar, a stereo camera, etc.). The three-dimensional point cloud collection device can collect the three-dimensional point cloud, and a three-dimensional model of a target room can be constructed by using the three-dimensional point cloud. The three-dimensional point cloud comprises a top surface point cloud, a ground surface point cloud and a placed article point cloud. The top surface point cloud is used for representing a ceiling of a target room, the ground surface point cloud is used for representing a ground of the target room, and the put article point cloud is used for representing various articles placed in the target room. In general, in the model of the target room, the point cloud located on the top surface may be used as the top surface point cloud, the point cloud located on the ground may be used as the ground surface point cloud, and the point cloud located inside the model space may be used as the item placement point cloud. It should be noted that the three-dimensional point cloud may also include a point cloud of a vertical plane, which is used to represent a plane perpendicular to the top surface or the ground, such as a wall surface, a vertical plane on which articles are placed, and the like.
In general, the Y axis of the coordinate system is oriented downward, the XOZ plane is a horizontal plane, and the normal vector of each plane is (nx, ny, nz). Ny for the top surface>Tceil(ii) a For the ground, ny<Tfloor(ii) a For the vertical plane, nx2+nz2>Twall，TceilCan be 0.985, TfloorCan be-0.985, TwallCan be 0.985.
In this embodiment, the generation module 1002 may generate a top point cloud projection image based on the top point cloud and a ground point cloud and a put item point cloud based on the ground point cloud and the put item point cloud. As an example, the generating module 1002 may project the area covered by the top point cloud onto a horizontal plane on which the top surface of the model of the target room is located, and then map the projected area on the horizontal plane onto a first image with a preset size according to a preset ratio. The generating module 1002 may further project the area covered by the ground point cloud and the area covered by the article placement point cloud onto a horizontal plane where the ground of the model of the target room is located, and then map the projected area on the horizontal plane onto a second image with a preset size according to a preset ratio.
In this embodiment, the determining module 1003 may determine at least one point cloud area to be compensated from the top point cloud projection image and the ground point cloud projection image. Specifically, as an example, the determining module 1003 may determine, from the top surface point cloud projection image and the ground point cloud projection image, an area to which the three-dimensional point cloud is not mapped as an area to be complemented.
In this embodiment, for each point cloud area to be compensated in the at least one point cloud area to be compensated, the compensation module 1004 may utilize the point clouds around the point cloud area to be compensated to compensate the point cloud area to be compensated based on the position of the point cloud area to be compensated. Specifically, for a certain point cloud area to be supplemented, the position of the point cloud area to be supplemented may be located on the top surface of the model of the target room, the ground surface, or the surface on which the object is placed, and since the horizontal plane coordinates of the point cloud area to be supplemented are known (e.g., x coordinates and z coordinates), the supplementing module 1004 may assign the height (e.g., y coordinates) of the three-dimensional point cloud near the point cloud area to be supplemented to the height of each point in the point cloud area to be supplemented, or perform fitting by using the height of the three-dimensional point cloud near the point cloud area to be supplemented, to obtain the height of each point in the point cloud area to be supplemented.
Referring to fig. 11, fig. 11 is a schematic structural diagram of a three-dimensional point cloud completion apparatus according to another exemplary embodiment of the present disclosure.
In some optional implementations, the generating module 1002 may include: a first generating unit 10021, configured to map the top point cloud onto a first image with a preset size and a preset color based on coordinates of points included in the top point cloud, so as to obtain a top point cloud projection image; the second generating unit 10022 is configured to map the ground point cloud and the other point clouds onto a second image with a preset size and a preset color based on coordinates of the ground point cloud and the other point clouds to obtain a ground point cloud projection image.
In some alternative implementations, the completion module 1004 may include: a supplementing unit 10041, configured to supplement, based on a preset point interval, a to-be-supplemented point set composed of multiple to-be-supplemented points into the to-be-supplemented point cloud region; a first determining unit 10042, configured to determine, by using the three-dimensional point cloud near the to-be-compensated point set, a height of each to-be-compensated point in the to-be-compensated point set; the completion unit 10043 is configured to complete the three-dimensional cavity corresponding to the point cloud area to be completed based on the height of each point to be completed.
In some optional implementations, the first determining unit 10042 may be further configured to: in response to determining that the point cloud area to be compensated is located on the top surface, for each point to be compensated in the point cloud area to be compensated, determining a point which is closest to the point to be compensated in the points included in the top surface point cloud; and determining the height of the determined point as the height of the point to be compensated.
In some optional implementations, the first determining unit 10042 may include: a first determining subunit 100421, configured to, in response to determining that the point cloud area to be compensated is located on the ground, determine, for each point to be compensated in a set of points to be compensated included in the point cloud area to be compensated, whether a vertical plane exists within a preset range of the point to be compensated based on the three-dimensional point cloud; a first dividing unit 100422, configured to, in response to determining that a vertical plane exists, divide the vertical plane into a plurality of partitions; and a second determining subunit 100423, configured to project the point to be compensated to the vertical plane, and determine the height of the block where the projected point is located as the height of the point to be compensated.
In some optional implementations, the first determining unit 10042 may further include: a second dividing subunit 100424, configured to, in response to determining that a vertical plane does not exist within the preset range of the point to be compensated, divide the set of points to be compensated included in the point cloud area to be compensated into at least one subset of points to be compensated; a third determining subunit 100425, configured to determine a peripheral to-be-compensated point subset from the at least one to-be-compensated point subset; a fourth determining subunit 100426, configured to, for each point to be compensated in the peripheral subset of points to be compensated, determine, from the three-dimensional point cloud, a point closest to the point to be compensated, and determine the height of the determined point as the height of the point to be compensated; a fifth determining subunit 100427, configured to determine the subset of peripheral points to be completed after determining the height as the reference point set; a complement subunit 100428, configured to re-determine a peripheral subset of points to be complemented from the at least one subset of points to be complemented, and based on the re-determined peripheral subset of points to be complemented, perform the following complementing steps: for each point to be compensated in the peripheral point to be compensated subset, determining the distance weight between the point to be compensated and each point in the reference point set, and performing weighted summation on the heights of the points in the reference point set by using the distance weight to obtain the height of the point to be compensated; adding the point to be compensated as a reference point into a reference point set; a sixth determining subunit 100429, configured to, in response to determining that at least one to-be-compensated point subset includes a to-be-compensated point subset that is not subjected to compensation, re-determine a peripheral to-be-compensated point subset from the to-be-compensated point subset that is not subjected to compensation, and continue to perform the compensation step using the re-determined peripheral to-be-compensated point subset.
In some alternative implementations, the second partitioning subunit 100424 may be further configured to: the following partitioning steps are performed: determining the number of points to be compensated in a circle which takes the points to be compensated as the center of the circle and takes the point distance as the radius for each point to be compensated in the point cloud area to be compensated; determining the points to be compensated corresponding to the circle centers of circles of which the number of the points to be compensated is less than the preset number as peripheral points to be compensated; determining each determined peripheral point to be supplemented as a subset of the points to be supplemented and removing the points from the set of the points to be supplemented; determining whether the number of the points to be compensated which are remained after removing the subset of the points to be compensated is zero or not; if the number is zero, ending the dividing step; and if the number is not zero, taking the residual points to be compensated after the point subset to be compensated is removed as a new point set to be compensated, and continuing to execute the dividing step.
In some optional implementations, the determining module 1003 may include: a second determining unit 10031, configured to determine at least one candidate point cloud area to be complemented from the top point cloud projection image and the ground point cloud projection image; the filtering unit 10032 is configured to filter a candidate point cloud area to be compensated that does not meet a preset condition in the at least one candidate point cloud area to be compensated, so as to obtain the at least one point cloud area to be compensated.
In some alternative implementations, the preset condition may include at least one of: the length-width ratio of a circumscribed rectangle of the candidate region to be compensated is within a preset proportion range; the area of the candidate region to be compensated is within a preset area range; the area ratio of the candidate to-be-compensated area to the circumscribed rectangle is larger than or equal to a preset ratio.
According to the three-dimensional point cloud complementing device provided by the embodiment of the disclosure, the top point cloud projection image and the ground point cloud projection image are generated, at least one point cloud area to be complemented is determined from the top point cloud projection image and the ground point cloud projection image, and finally the point cloud area to be complemented is complemented by utilizing the point cloud around the point cloud area to be complemented based on the position of the point cloud area to be complemented, so that the position characteristics of the point cloud area to be complemented are effectively utilized, the point cloud area to be complemented is complemented, the accuracy of complementing the cavity in the three-dimensional point cloud is improved, and the integrity and the attractiveness of the house model are increased.
Exemplary computer program product and computer-readable storage Medium
In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the three-dimensional point cloud completion method according to various embodiments of the present disclosure described in the "exemplary methods" section above of this specification.
The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps in the three-dimensional point cloud completion method according to various embodiments of the present disclosure described in the "exemplary methods" section above in this specification.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Exemplary three-dimensional Point cloud completion apparatus
Next, a three-dimensional point cloud complementing apparatus according to an embodiment of the present disclosure is described with reference to fig. 12. The three-dimensional point cloud complementing device may be either or both of the terminal device 101 and the server 103 as shown in fig. 1, or a stand-alone device independent thereof, which may communicate with the terminal device 101 and the server 103 to receive the acquired input signals therefrom.
Fig. 12 illustrates a block diagram of a three-dimensional point cloud completion apparatus according to an embodiment of the present disclosure.
As shown in fig. 12, the three-dimensional point cloud complementing apparatus 1200 includes one or more processors 1201 and a memory 1202.
The processor 1201 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the three-dimensional point cloud replenishment apparatus 1200 to perform desired functions.
Memory 1202 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory can include, for example, Random Access Memory (RAM), cache memory (or the like). The non-volatile memory may include, for example, Read Only Memory (ROM), a hard disk, flash memory, and the like. One or more computer program instructions may be stored on a computer readable storage medium and executed by the processor 1201 to implement the three-dimensional point cloud completion methods of the various embodiments of the present disclosure above and/or other desired functions. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.
In one example, the three-dimensional point cloud complementing apparatus 1200 may further include: an input device 1203 and an output device 1204, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
For example, when the three-dimensional point cloud complementing device is the terminal device 101 or the server 103, the input device 1203 may be a mouse, a keyboard, a laser radar, or the like, and is configured to input the three-dimensional point cloud. When the three-dimensional point cloud complementing device is a stand-alone device, the input device 1203 may be a communication network connector for receiving the input three-dimensional point cloud from the terminal device 101 and the server 103.
The output device 1204 may output various information including a complemented three-dimensional point cloud to the outside. The output devices 1204 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, among others.
Of course, for simplicity, only some of the components of the three-dimensional point cloud complementing device 1200 relevant to the present disclosure are shown in fig. 12, and components such as buses, input/output interfaces, and the like are omitted. In addition, the three-dimensional point cloud complementing device 1200 may include any other suitable components according to specific applications.
The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.
In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts in the embodiments are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
The block diagrams of devices, apparatuses, systems referred to in this disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.
It is also noted that in the devices, apparatuses, and methods of the present disclosure, each component or step can be decomposed and/or recombined. These decompositions and/or recombinations are to be considered equivalents of the present disclosure.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.

Claims (10)

1. A three-dimensional point cloud completion method comprises the following steps:
acquiring a three-dimensional point cloud corresponding to a target room, wherein the three-dimensional point cloud comprises a top surface point cloud, a ground point cloud and a put article point cloud;
generating a top surface point cloud projection image based on the top surface point cloud, and generating a ground point cloud projection image based on the ground point cloud and the put item point cloud;
determining at least one point cloud area to be supplemented from the top surface point cloud projection image and the ground point cloud projection image;
and for each point cloud area to be compensated in the at least one point cloud area to be compensated, based on the position of the point cloud area to be compensated, utilizing the point clouds around the point cloud area to be compensated to compensate the point cloud area to be compensated.
2. The method of claim 1, wherein the generating a top surface point cloud projection image based on the top surface point cloud and generating a ground point cloud projection image based on the ground point cloud and the posed article point cloud comprises:
based on coordinates of points included in the top surface point cloud, mapping the top surface point cloud to a first image with a preset size and a preset color to obtain a top surface point cloud projection image;
and mapping the ground point cloud and the other point clouds onto a second image with a preset size and a preset color based on the coordinates of the ground point cloud and the other point clouds to obtain a ground point cloud projection image.
3. The method of claim 1, wherein the complementing the point cloud area to be complemented with the point clouds around the point cloud area to be complemented based on the position of the point cloud area to be complemented comprises:
supplementing a point set to be supplemented, which consists of a plurality of points to be supplemented, into the point cloud area to be supplemented based on a preset point interval;
determining the height of each point to be supplemented in the point set to be supplemented by utilizing the three-dimensional point cloud near the point set to be supplemented;
and completing the three-dimensional cavity corresponding to the point cloud area to be completed based on the height of each point to be completed.
4. The method of claim 3, wherein the determining the height of each point to be complemented in the set of points to be complemented using the three-dimensional point cloud in the vicinity of the set of points to be complemented comprises:
in response to determining that the point cloud area to be compensated is located on the top surface, for each point to be compensated in the point cloud area to be compensated, determining a point which is closest to the point to be compensated in the points included in the point cloud of the top surface; and determining the height of the determined point as the height of the point to be compensated.
5. The method of claim 3, wherein the determining the height of each point to be complemented in the set of points to be complemented using the three-dimensional point cloud in the vicinity of the set of points to be complemented comprises:
in response to the fact that the point cloud area to be supplemented is located on the ground, determining whether a vertical plane exists in a preset range of the point to be supplemented or not for each point to be supplemented in a point set to be supplemented included in the point cloud area to be supplemented based on the three-dimensional point cloud;
in response to determining that a vertical plane exists, dividing the vertical plane to obtain a plurality of partitions;
and projecting the point to be compensated to the vertical plane, and determining the height of the block where the projected point is located as the height of the point to be compensated.
6. The method of claim 5, wherein the determining a height of each point to be complemented in the set of points to be complemented using a three-dimensional point cloud in a vicinity of the set of points to be complemented further comprises:
in response to the fact that the vertical plane does not exist in the preset range of the point to be compensated, dividing a point set to be compensated, which is included in the point cloud area to be compensated, into at least one point subset to be compensated;
determining a peripheral point subset to be complemented from the at least one point subset to be complemented;
for each point to be compensated in the peripheral point to be compensated subset, determining a point closest to the point to be compensated from the three-dimensional point cloud, and determining the height of the determined point as the height of the point to be compensated;
determining the peripheral point subset to be compensated after the height is determined as a reference point set;
re-determining a peripheral point subset to be supplemented from the at least one point subset to be supplemented, and executing the following supplementing steps based on the re-determined peripheral point subset to be supplemented:
for each point to be compensated in the peripheral point to be compensated subset, determining the distance weight between the point to be compensated and each point in the reference point set, and performing weighted summation on the heights of the points in the reference point set by using the distance weight to obtain the height of the point to be compensated; adding the point to be compensated as a reference point into the reference point set;
and in response to determining that the at least one to-be-supplemented point subset has a to-be-supplemented point subset which is not subjected to supplementation, re-determining a peripheral to-be-supplemented point subset from the to-be-supplemented point subset, and continuing to execute the supplementing step by using the re-determined peripheral to-be-supplemented point subset.
7. The method of claim 6, wherein the dividing the set of points to be compensated included in the point cloud region to be compensated into at least one subset of points to be compensated comprises:
the following partitioning steps are performed:
determining the number of points to be compensated in a circle which takes the points to be compensated as the center of the circle and the point distance as the radius for each point to be compensated in the point cloud area to be compensated which comprises the point set to be compensated;
determining the points to be compensated corresponding to the circle centers of circles of which the number of the points to be compensated is less than the preset number as peripheral points to be compensated;
determining each determined peripheral point to be supplemented as a subset of points to be supplemented and removing the points from the set of points to be supplemented;
determining whether the number of the points to be compensated which are remained after the point subset to be compensated is removed is zero or not;
if the number is zero, ending the dividing step;
and if the number is not zero, taking the residual points to be compensated after the point subset to be compensated is removed as a new point set to be compensated, and continuously executing the dividing step.
8. A three-dimensional point cloud completion apparatus, comprising:
the system comprises an acquisition module, a storage module and a display module, wherein the acquisition module is used for acquiring three-dimensional point clouds corresponding to a target room, and the three-dimensional point clouds comprise a top surface point cloud, a ground point cloud and a put article point cloud;
a generation module for generating a top point cloud projection image based on the top point cloud, and generating a ground point cloud projection image based on the ground point cloud and the put item point cloud;
the determining module is used for determining at least one point cloud area to be supplemented from the top surface point cloud projection image and the ground point cloud projection image;
and the complementing module is used for complementing each point cloud area to be complemented in the at least one point cloud area to be complemented by utilizing the point clouds around the point cloud area to be complemented based on the position of the point cloud area to be complemented.
9. A computer-readable storage medium, the storage medium storing a computer program for performing the method of any of the preceding claims 1-7.
10. A three-dimensional point cloud complementing apparatus, comprising:
at least one processor;
a memory coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor, the at least one processor implementing the method of any one of claims 1 to 7 by executing the instructions stored by the memory.
CN202010257587.XA 2020-04-03 2020-04-03 Three-dimensional point cloud completion method, device and computer readable storage medium Active CN111383355B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010257587.XA CN111383355B (en) 2020-04-03 2020-04-03 Three-dimensional point cloud completion method, device and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010257587.XA CN111383355B (en) 2020-04-03 2020-04-03 Three-dimensional point cloud completion method, device and computer readable storage medium

Publications (2)

Publication Number Publication Date
CN111383355A true CN111383355A (en) 2020-07-07
CN111383355B CN111383355B (en) 2023-06-27

Family

ID=71218955

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010257587.XA Active CN111383355B (en) 2020-04-03 2020-04-03 Three-dimensional point cloud completion method, device and computer readable storage medium

Country Status (1)

Country Link
CN (1) CN111383355B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112700537A (en) * 2020-12-31 2021-04-23 广东美的白色家电技术创新中心有限公司 Tire point cloud construction method, tire point cloud assembly method, tire point cloud control device, and storage medium
CN113052955A (en) * 2021-03-19 2021-06-29 西安电子科技大学 Point cloud completion method, system and application
CN113192201A (en) * 2021-05-08 2021-07-30 上海皓桦科技股份有限公司 Data fitting method, device and medium for point cloud data
CN113378270A (en) * 2021-06-10 2021-09-10 北京千丁互联科技有限公司 House structure analysis method and device, terminal equipment and readable storage medium

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102436654A (en) * 2011-09-02 2012-05-02 清华大学 Adaptive segmentation method of building point cloud
CN104063898A (en) * 2014-06-30 2014-09-24 厦门大学 Three-dimensional point cloud auto-completion method
CN104484668A (en) * 2015-01-19 2015-04-01 武汉大学 Unmanned aerial vehicle multi-overlapped-remote-sensing-image method for extracting building contour line
CN104931953A (en) * 2015-06-30 2015-09-23 天津大学 Quad-rotor unmanned plane three-dimension environment reconstruction method based on two-dimension laser radar
CN107223268A (en) * 2015-12-30 2017-09-29 中国科学院深圳先进技术研究院 A kind of three-dimensional point cloud model method for reconstructing and device
CN108053473A (en) * 2017-12-29 2018-05-18 北京领航视觉科技有限公司 A kind of processing method of interior three-dimensional modeling data
CN108541322A (en) * 2016-08-29 2018-09-14 北京清影机器视觉技术有限公司 The treating method and apparatus of dimensional visual measurement data
CN108765535A (en) * 2018-05-30 2018-11-06 链家网（北京）科技有限公司 The labeling method and device in a kind of three-dimensional point cloud missing data region
CN109711018A (en) * 2018-12-15 2019-05-03 武汉兴联云立方科技有限公司 A kind of Size Dwelling Design method of 2 d-to-3 d
CN110349257A (en) * 2019-07-16 2019-10-18 四川大学 A kind of binocular measurement missing point cloud interpolating method based on the mapping of phase puppet
CN110869974A (en) * 2018-11-19 2020-03-06 深圳市大疆创新科技有限公司 Point cloud processing method, point cloud processing device and storage medium
EP3620941A1 (en) * 2018-09-05 2020-03-11 3Frog Nv Generating a spatial model of an indoor structure

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102436654A (en) * 2011-09-02 2012-05-02 清华大学 Adaptive segmentation method of building point cloud
CN104063898A (en) * 2014-06-30 2014-09-24 厦门大学 Three-dimensional point cloud auto-completion method
CN104484668A (en) * 2015-01-19 2015-04-01 武汉大学 Unmanned aerial vehicle multi-overlapped-remote-sensing-image method for extracting building contour line
CN104931953A (en) * 2015-06-30 2015-09-23 天津大学 Quad-rotor unmanned plane three-dimension environment reconstruction method based on two-dimension laser radar
CN107223268A (en) * 2015-12-30 2017-09-29 中国科学院深圳先进技术研究院 A kind of three-dimensional point cloud model method for reconstructing and device
CN108541322A (en) * 2016-08-29 2018-09-14 北京清影机器视觉技术有限公司 The treating method and apparatus of dimensional visual measurement data
CN108053473A (en) * 2017-12-29 2018-05-18 北京领航视觉科技有限公司 A kind of processing method of interior three-dimensional modeling data
CN108765535A (en) * 2018-05-30 2018-11-06 链家网（北京）科技有限公司 The labeling method and device in a kind of three-dimensional point cloud missing data region
EP3620941A1 (en) * 2018-09-05 2020-03-11 3Frog Nv Generating a spatial model of an indoor structure
CN110869974A (en) * 2018-11-19 2020-03-06 深圳市大疆创新科技有限公司 Point cloud processing method, point cloud processing device and storage medium
CN109711018A (en) * 2018-12-15 2019-05-03 武汉兴联云立方科技有限公司 A kind of Size Dwelling Design method of 2 d-to-3 d
CN110349257A (en) * 2019-07-16 2019-10-18 四川大学 A kind of binocular measurement missing point cloud interpolating method based on the mapping of phase puppet

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112700537A (en) * 2020-12-31 2021-04-23 广东美的白色家电技术创新中心有限公司 Tire point cloud construction method, tire point cloud assembly method, tire point cloud control device, and storage medium
CN113052955A (en) * 2021-03-19 2021-06-29 西安电子科技大学 Point cloud completion method, system and application
CN113052955B (en) * 2021-03-19 2023-06-30 西安电子科技大学 Point cloud completion method, system and application
CN113192201A (en) * 2021-05-08 2021-07-30 上海皓桦科技股份有限公司 Data fitting method, device and medium for point cloud data
CN113192201B (en) * 2021-05-08 2023-08-01 上海皓桦科技股份有限公司 Data fitting method, device and medium of point cloud data
CN113378270A (en) * 2021-06-10 2021-09-10 北京千丁互联科技有限公司 House structure analysis method and device, terminal equipment and readable storage medium
CN113378270B (en) * 2021-06-10 2023-07-28 北京千丁互联科技有限公司 House structure analysis method, device, terminal equipment and readable storage medium

Also Published As

Publication number Publication date
CN111383355B (en) 2023-06-27

Similar Documents

Publication Publication Date Title
CN111383355A (en) Three-dimensional point cloud completion method and device and computer readable storage medium
CN112767551B (en) Three-dimensional model construction method and device, electronic equipment and storage medium
CN111985036A (en) House type frame line drawing method and device, storage medium and electronic equipment
CN113077548B (en) Collision detection method, device, equipment and storage medium for object
CN113989376B (en) Method and device for acquiring indoor depth information and readable storage medium
CN111402404B (en) Panorama complementing method and device, computer readable storage medium and electronic equipment
CN112907755B (en) Model display method and device in three-dimensional house model
CN111681320B (en) Model display method and device in three-dimensional house model
CN110478898B (en) Configuration method and device of virtual scene in game, storage medium and electronic equipment
CN112489114A (en) Image conversion method and device, computer readable storage medium and electronic equipment
WO2023038990A1 (en) Methods and apparatuses for calculating building heights from mono imagery
CN114820979B (en) Processing method and device of three-dimensional grid model and storage medium
JP2022551249A (en) IMAGE PROCESSING METHOD, IMAGE PROCESSING COMMAND GENERATION METHOD AND APPARATUS
CN114170381A (en) Three-dimensional path display method and device, readable storage medium and electronic equipment
US11706395B2 (en) Apparatus and method for selecting camera providing input images to synthesize virtual view images
CN112950759B (en) Three-dimensional house model construction method and device based on house panoramic image
CN111080792B (en) Model simplification processing method and device, electronic equipment and storage medium
CN113689508A (en) Point cloud marking method and device, storage medium and electronic equipment
CN115619989B (en) Fusion effect diagram generation method and device, electronic equipment and storage medium
CN112465971A (en) Method and device for guiding point positions in model, storage medium and electronic equipment
CN115063564B (en) Method, device and medium for displaying object label in two-dimensional display image
CN116342785A (en) Image processing method, device, equipment and medium
CN115512046A (en) Panorama display method and device for model outer point positions, equipment and medium
WO2020117385A1 (en) Hybrid image rendering system
CN113762173B (en) Training method and device for face optical flow estimation and optical flow value prediction model

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20200921

Address after: 100085 Floor 102-1, Building No. 35, West Second Banner Road, Haidian District, Beijing

Applicant after: Seashell Housing (Beijing) Technology Co.,Ltd.

Address before: 300 457 days Unit 5, Room 1, 112, Room 1, Office Building C, Nangang Industrial Zone, Binhai New Area Economic and Technological Development Zone, Tianjin

Applicant before: BEIKE TECHNOLOGY Co.,Ltd.

TA01 Transfer of patent application right
TA01 Transfer of patent application right

Effective date of registration: 20220325

Address after: 100085 8th floor, building 1, Hongyuan Shouzhu building, Shangdi 6th Street, Haidian District, Beijing

Applicant after: As you can see (Beijing) Technology Co.,Ltd.

Address before: 100085 Floor 101 102-1, No. 35 Building, No. 2 Hospital, Xierqi West Road, Haidian District, Beijing

Applicant before: Seashell Housing (Beijing) Technology Co.,Ltd.

GR01 Patent grant
GR01 Patent grant