CN111583392B - Object three-dimensional reconstruction method and system - Google Patents
Object three-dimensional reconstruction method and system Download PDFInfo
- Publication number
- CN111583392B CN111583392B CN202010358037.7A CN202010358037A CN111583392B CN 111583392 B CN111583392 B CN 111583392B CN 202010358037 A CN202010358037 A CN 202010358037A CN 111583392 B CN111583392 B CN 111583392B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- data
- cloud data
- interest
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
Abstract
The invention provides a three-dimensional reconstruction method and a three-dimensional reconstruction system for an object.A processor receives original three-dimensional point cloud data, and determines whether a first point cloud point corresponding to the first point cloud data is in an area of interest corresponding to preset area of interest data according to the first point cloud data and the preset area of interest data; when the first point cloud point is in the region of interest, storing the first point cloud data in the three-dimensional point cloud data of the object; when the first point cloud point is not in the region of interest, determining first mirror equation data, carrying out reflection conversion processing according to the first point cloud data and the first mirror equation data to obtain first reflection point cloud data, and generating second reflection point cloud data when the first reflection point cloud point is determined to be in the region of interest; and when the first reflection point cloud data is determined to be reliable point cloud data according to the second reflection point cloud data and the camera center position data, storing the first reflection point cloud data in the three-dimensional point cloud data of the object.
Description
Technical Field
The invention relates to the field of data processing, in particular to a three-dimensional object reconstruction method and system.
Background
With the development of computer technology and the rapid development of data processing in recent years, 3D modeling of objects (e.g., objects of interest) from image data is a popular subject of computer vision. Reconstructing a three-dimensional model is an important process in many fields such as computer animation, medical imaging, computer graphics, and the like.
Reconstruction is performed using a series of images taken from the camera at different positions. One obvious disadvantage of this approach is that the object of interest must be static. Such a system is not suitable for working on dynamic objects and requires a priori knowledge such as the rotational speed of the turntable. Another approach is to use a method that performs shape extraction from the contours with the support of multiple cameras to retrieve the visual shell of the object, but this method lacks a concave region in the reconstructed model. Still other solutions employ a system of multi-color cameras and depth sensors, which has the major disadvantage of requiring a synchronization protocol to work on moving objects, sometimes each camera and sensor must be connected to a unique computer, which can achieve good accuracy, but is costly in equipment and creates unnecessary resource redundancy in the process. Furthermore, the use of multiple depth cameras may result in severe infrared interference.
Disclosure of Invention
Aiming at the defects of the prior art, the embodiment of the invention aims to provide a three-dimensional object reconstruction method and system for reconstructing a three-dimensional object model of an object through analysis and calculation processing by shooting an image of a target scene where the object is located.
To solve the above problems, in a first aspect, the present invention provides a three-dimensional reconstruction method of an object, the method comprising:
the processor receives original three-dimensional point cloud data sent by the three-dimensional point cloud imaging device; wherein the original three-dimensional point cloud data comprises a plurality of first point cloud data;
performing position judgment processing according to the first point cloud data and preset region of interest data, and determining whether a first point cloud point corresponding to the first point cloud data is in a region of interest corresponding to the preset region of interest data;
when the first point cloud point is in the region of interest, determining the first point cloud data as reliable point cloud data, and storing the first point cloud data in object three-dimensional point cloud data;
when the first point cloud point is not in the region of interest, determining first mirror equation data according to the first point cloud data, preset mirror equation data and the preset region of interest data;
performing reflection transformation processing according to the first point cloud data and the first mirror equation data to obtain first reflection point cloud data;
position judgment processing is carried out according to the first reflection point cloud data and the preset region of interest data, and whether a first reflection point cloud point corresponding to the first reflection point cloud data is in the region of interest is determined;
when the first reflection point cloud point is in the region of interest, carrying out reflection transformation processing according to the first reflection point cloud data and the second mirror equation data to obtain second reflection point cloud data;
determining whether the first reflected point cloud data is reliable point cloud data according to the first point cloud data, the second reflected point cloud data and preset camera center position data;
and when the first reflection point cloud data is reliable point cloud data, storing the first reflection point cloud data in the three-dimensional point cloud data of the object.
Preferably, the determining whether the first point cloud point corresponding to the first point cloud data is in the region of interest corresponding to the preset region of interest data specifically includes:
according to the first point cloud data and the preset region of interest data, determining coordinate values of projection points of the first point cloud points on a plane of interest corresponding to the preset region of interest data;
determining whether the projection point is in an area of interest corresponding to the preset area of interest data by adopting a ray method;
when the projection point is in the region of interest, determining that the first cloud point is in the region of interest; and when the projection point is not in the region of interest, determining that the first cloud point is not in the region of interest.
Preferably, the preset mirror equation data includes two sets of mirror equation data, and the determining the first mirror equation data according to the first point cloud data, the preset mirror equation data, and the preset region of interest data specifically includes:
determining a group of mirror equation data in the preset mirror equation data as judging mirror equation data;
performing reflection transformation processing according to the first point cloud data and the determined mirror equation data to obtain reflected point cloud data;
performing position judgment processing according to the reflection point cloud data and the preset region of interest data, and determining whether the reflection point cloud point corresponding to the reflection point cloud data is in the region of interest;
and when the reflection point cloud point is in the region of interest, determining the determined mirror equation data as first mirror equation data, otherwise, determining the other set of mirror equation data in the preset mirror equation data as first mirror equation data.
Preferably, the determining whether the first reflected point cloud data is reliable point cloud data according to the first point cloud data, the second reflected point cloud data and the preset camera center position data specifically includes:
performing distance calculation processing according to the first point cloud data and preset camera center position data to obtain first distance data;
performing distance calculation processing according to the second reflection point cloud data and the preset camera center position data to obtain second distance data;
and when the first distance data is smaller than the second distance data, determining the first reflection point cloud data as reliable point cloud data.
Preferably, before the processor receives the original three-dimensional point cloud data sent by the three-dimensional point cloud imaging device, the method further includes:
the three-dimensional point cloud imaging device receives an externally input image acquisition instruction;
shooting a target scene according to the image acquisition instruction, and generating original three-dimensional point cloud data of the target scene;
the three-dimensional point cloud imaging device sends the original three-dimensional point cloud data to the processor.
Further preferably, the three-dimensional point cloud imaging device is a time-of-flight camera.
Preferably, the method further comprises:
the processor sends the three-dimensional point cloud data of the object to a display device;
and the display equipment displays and outputs according to the three-dimensional point cloud data of the object.
In a second aspect, the present invention provides a three-dimensional reconstruction system for an object, the system comprising: the three-dimensional point cloud imaging device, the first reflecting device, the second reflecting device and the processor;
the three-dimensional point cloud imaging device is used for receiving an externally input image acquisition instruction, shooting a target scene according to the image acquisition instruction and generating original three-dimensional point cloud data of the target scene;
the first reflecting device is used for reflecting the light emitted by the three-dimensional point cloud imaging device to the surface of the object and secondarily reflecting the light reflected by the surface of the object to the three-dimensional point cloud imaging device;
the second reflecting device is used for reflecting the light emitted by the three-dimensional point cloud imaging device to the surface of the object and secondarily reflecting the light reflected by the surface of the object to the three-dimensional point cloud imaging device;
the processor is used for receiving the original three-dimensional point cloud data sent by the three-dimensional point cloud imaging device; wherein the original three-dimensional point cloud data comprises a plurality of first point cloud data;
the processor is further configured to perform position judgment according to the first point cloud data and preset region of interest data, and determine whether a first point cloud point corresponding to the first point cloud data is in a region of interest corresponding to the preset region of interest data;
when the first point cloud point is in the region of interest, the processor is further configured to determine the first point cloud data as reliable point cloud data, and store the first point cloud data in object three-dimensional point cloud data;
when the first point cloud point is not in the region of interest, the processor is further configured to determine first mirror equation data according to the first point cloud data, preset mirror equation data, and the preset region of interest data;
the processor is further used for carrying out reflection transformation processing according to the first point cloud data and the first mirror equation data to obtain first reflection point cloud data;
the processor is further configured to perform position judgment processing according to the first reflection point cloud data and the preset region of interest data, and determine whether a first reflection point cloud point corresponding to the first reflection point cloud data is in the region of interest;
when the first reflection point cloud point is in the region of interest, the processor is further configured to perform reflection transformation processing according to the first reflection point cloud data and the second mirror equation data to obtain second reflection point cloud data;
the processor is further configured to determine whether the first reflected point cloud data is reliable point cloud data according to the first point cloud data, the second reflected point cloud data, and preset camera center position data;
and when the first reflection point cloud data is reliable point cloud data, the processor is further configured to store the first reflection point cloud data in the three-dimensional point cloud data of the object.
Preferably, the system further comprises a display device:
the display device is used for receiving the three-dimensional point cloud data of the object sent by the processor;
the display device is also used for displaying and outputting according to the three-dimensional point cloud data of the object.
Further preferably, the system further comprises:
the three-dimensional point cloud imaging device is in communication connection with the processor in a wired or wireless mode;
the processor is in communication connection with the display device through a wired or wireless communication mode.
According to the object three-dimensional reconstruction method provided by the embodiment of the invention, a frame of image of a target scene is acquired by using a time-of-flight camera, original three-dimensional point cloud data is generated, the acquired and generated original three-dimensional point cloud data is combined with preset scene information data to analyze, reliable point cloud data in the original three-dimensional point cloud data is determined, the reliable point cloud data is generated through point cloud points in the original three-dimensional point cloud data, and finally, a plurality of the reliable point cloud data determined from the original three-dimensional point cloud and the generated reliable point cloud data jointly form object three-dimensional point cloud data, so that three-dimensional point cloud reconstruction of an object is completed. The method provided by the embodiment of the invention has simple algorithm and corresponding system structure, and can finish the reconstruction of the three-dimensional point cloud of the dynamic and static objects on the basis of reducing the equipment cost.
Drawings
FIG. 1 is a schematic diagram of an object three-dimensional reconstruction system according to an embodiment of the present invention;
fig. 2 is a flowchart of a three-dimensional reconstruction method for an object according to an embodiment of the present invention.
Detailed Description
The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
In order to better explain the object three-dimensional reconstruction method provided by the embodiment of the invention, firstly, a description is given to an object three-dimensional reconstruction system provided by the embodiment of the invention.
Fig. 1 is a schematic diagram of an object three-dimensional reconstruction system according to an embodiment of the present invention, as shown in the drawing, where the object three-dimensional reconstruction system according to the embodiment of the present invention includes: the three-dimensional point cloud imaging device 1, the first reflecting device 2, the second reflecting device 3, the processor 4 and the display apparatus 5.
The three-dimensional point cloud imaging device 1 is used for receiving an externally input image acquisition instruction, shooting a target scene according to the image acquisition instruction, and generating original three-dimensional point cloud data of the target scene.
The first reflection device 2 is used for reflecting the light emitted by the three-dimensional point cloud imaging device 1 to the surface of the object and secondarily reflecting the light reflected by the surface of the object to the three-dimensional point cloud imaging device 1. The light emitted by the three-dimensional point cloud imaging device can be structural light or other forms of light.
And a second reflecting means 3 for reflecting the light emitted from the three-dimensional point cloud imaging means 1 to the object surface and secondarily reflecting the light reflected from the object surface to the three-dimensional point cloud imaging means 1. In a preferred embodiment of the present invention, the three-dimensional point cloud imaging apparatus 1 is a time-of-flight camera. The light emitted by the three-dimensional point cloud imaging device can be structured light or unstructured light.
The processor 4 is configured to receive the original three-dimensional point cloud data sent by the three-dimensional point cloud imaging device, determine reliable point cloud data in the original three-dimensional point cloud data according to the original three-dimensional point data, the preset region of interest data, the preset mirror equation data and the preset camera center position data, generate reliable point cloud data according to the original three-dimensional point cloud data, and store the reliable point cloud data in the three-dimensional point cloud data of the object, thereby completing three-dimensional reconstruction of the object. The processor is further configured to send the three-dimensional point cloud data of the object to the display device 5.
And the display device 5 is used for receiving the three-dimensional object point cloud data sent by the processor and displaying and outputting according to the three-dimensional object point cloud data. In a preferred embodiment of the present invention, the display device 5 may be any electronic display device capable of performing display output on a three-dimensional point cloud.
In a preferred embodiment of the present invention, the three-dimensional point cloud imaging apparatus 1 is in communication connection with the processor 4 through a wired or wireless manner, and the processor 4 is in communication connection with the display device 5 through a wired or wireless communication manner.
In a preferred embodiment of the present invention, the first reflecting device 2 and the second reflecting device 3 are disposed in front of the three-dimensional point cloud imaging device 1, and the object is located in a target space range formed by the three-dimensional point cloud imaging device 1, the first reflecting device 2 and the second reflecting device 3. In a specific example of the embodiment of the present invention, the planes of the first reflecting device 2 and the second reflecting device 3 intersect at an intersection line, and the object is placed at a plane position formed by connecting the central point of the three-dimensional point cloud imaging device 1 and the intersection line, that is, the object is placed centrally, wherein the centering is not an accurate position, and only the centering is needed to be approximately performed.
Before the object three-dimensional reconstruction system provided by the embodiment of the invention is used, the three-dimensional point cloud imaging device 1, the first reflecting device 2, the second reflecting device 3 and the object placement position in the system are adjusted, so that the system can be positioned at the best data acquisition position. For example, the angle between the first reflecting means 2 and the second reflecting means 3 is adjusted. Or the distance between the three-dimensional point cloud imaging device 1 and the first reflecting device 2 and the distance between the three-dimensional point cloud imaging device and the second reflecting device 3 are adjusted. After the adjustment of the three-dimensional point cloud imaging device 1, the first reflecting device 2 and the second reflecting device 3 is completed, mirror equation data of the first reflecting device 2 and the second reflecting device 3 are determined, and region of interest data and preset camera center position data are determined.
The above describes an object three-dimensional reconstruction system provided by the embodiment of the present invention in detail, and the following describes an object three-dimensional reconstruction method provided by the embodiment of the present invention based on the object three-dimensional reconstruction system provided by the embodiment of the present invention in detail.
Fig. 2 is a flowchart of a three-dimensional reconstruction method for an object according to an embodiment of the present invention, as shown in the drawings, the method specifically includes the following steps:
Wherein the original three-dimensional point cloud data includes a plurality of first point cloud data.
Specifically, the original three-dimensional point cloud data is generated after the three-dimensional point cloud imaging device shoots the target scene. In a preferred scheme of the embodiment of the invention, the three-dimensional point cloud imaging device is a time-of-flight camera, and the original three-dimensional point cloud data is generated after the time-of-flight camera collects a target scene.
In the preferred scheme of the embodiment of the invention, when the three-dimensional target point cloud reconstruction is required to be carried out on the object, the object is placed in a target scene constructed by the object three-dimensional reconstruction system provided by the embodiment of the invention. The three-dimensional point cloud imaging device receives an externally input image acquisition instruction, shoots a target scene according to the image acquisition instruction, generates original three-dimensional point cloud data of the target scene, and sends the original three-dimensional point cloud data to the processor. The externally input image acquisition instruction may be generated by a user pressing a photographing key of the time-of-flight camera or by a traffic device connected to the time-of-flight camera transmitting an image acquisition signal.
In the preferred scheme of the embodiment of the invention, the adopted flight time camera illumination module and the receiving module are composed of 4 infrared laser diodes with the emission wavelength of 850nm, and ground glass is additionally arranged at the front ends of the 4 diodes to expand the irradiation range of light beams. The receiving module is a CMOS pixel array with the resolution of 240 multiplied by 320, and the received photons sequentially reach each corresponding pixel point through the lens.
In another preferred embodiment of the present invention, a time-of-flight camera resolution of 512 x 424 pixels is used that is capable of acquisition at a maximum frequency of 30fps, with a measurable depth in the range of 0.5 meters to 4.5 meters, a horizontal viewing angle of 70 deg., and a vertical viewing angle of 60 deg.. And it emits infrared light with a wavelength of 790nm using continuous wave modulation.
Specifically, each first three-dimensional point cloud data in the original three-dimensional point cloud data is analyzed respectively. Firstly, judging according to three-dimensional coordinate values of the first three-dimensional point cloud data and preset region-of-interest data, and determining whether the first three-dimensional point cloud point is in the region of interest. The preset region of interest data are boundary data of limited polygons parallel to the camera plane in the target scene. When the first point cloud point is within the region of interest, step 103 is performed. When the first cloud point is not within the region of interest, steps 104 and thereafter are performed.
In a preferred embodiment of the present invention, determining whether the first three-dimensional point cloud point is in the region of interest includes: firstly, the processor determines coordinate values of projection points of first point cloud points on a plane of interest corresponding to preset region of interest data according to the first point cloud data and the preset region of interest data. Secondly, a preset judging method is called to determine whether the projection point is in the region of interest, and when the projection point is in the region of interest, the first cloud point is determined to be in the region of interest; when the projected point is not within the region of interest, it is determined that the first cloud point is not within the region of interest. In a specific example of the embodiment of the present invention, the preset determination method is a ray method.
And step 103, determining the first point cloud data as reliable point cloud data, and storing the first point cloud data in the three-dimensional point cloud data of the object.
In particular, reliable point cloud data refers to point cloud data points that can be used to reconstruct a three-dimensional point cloud of an object. The reliable point cloud data in this step is essentially point cloud data generated after an object directly acquired by the time-of-flight camera is facing a point on the side of the time-of-flight camera.
Specifically, the preset mirror equation data is mirror equation data corresponding to the first reflecting device and the second reflecting device determined through analysis and calculation after the position adjustment of the relevant components in the object three-dimensional reconstruction system provided by the embodiment of the invention is determined. In the embodiment of the invention, the first reflecting device and the second reflecting device have the same attribute and material. In a preferred embodiment of the present invention, the first reflecting device and the second reflecting device are plane mirrors. Therefore, the preset mirror equation in the embodiment of the invention comprises two sets of mirror equation data, which correspond to the first reflecting device and the second radiating device respectively. The purpose of determining the first specular equation data is to determine whether the first point cloud point corresponding to the first point cloud data is behind the first reflecting device or behind the second reflecting device. That is to say, it is determined whether the first point cloud data is point cloud data generated by the time-of-flight camera from light reflected by the acquired first reflecting means or point cloud data generated by the time-of-flight camera from light reflected by the acquired second reflecting means.
In a preferred embodiment of the present invention, determining the first mirror equation data according to the first point cloud data, the preset mirror equation data, and the region of interest data specifically includes the following steps:
first, a set of mirror equation data among the preset mirror equation data is determined as the determination mirror equation data. That is, any one of the two sets of mirror surface equation data is taken as the determination mirror surface equation data for making the determination.
And secondly, carrying out reflection transformation processing according to the first point cloud data and the determined mirror equation data to obtain reflection point cloud data. That is, according to the plane mirror imaging principle, three-dimensional coordinate values of the first point cloud data and the data of the judging mirror equation are calculated to generate data of points symmetrical to the judging mirror. Wherein data corresponding to points for which specular symmetry is determined is regarded as reflected point cloud data.
And finally, carrying out position judgment processing according to the reflection point cloud data and the preset region of interest data, and determining whether the reflection point cloud point corresponding to the reflection point cloud data is in the region of interest. And when the cloud point of the reflection point is in the region of interest, determining that the mirror equation data is determined to be the first mirror equation data, otherwise, determining that the other set of mirror equation data in the two sets of mirror equation data is determined to be the first mirror equation data.
And 105, performing reflection transformation processing according to the first point cloud data and the first mirror equation data to obtain first reflection point cloud data.
Specifically, according to the plane mirror imaging principle, the three-dimensional coordinate value of the first point cloud data and the first mirror equation data are calculated, and the data of the point symmetrical to the reflecting device corresponding to the first mirror equation data are generated. Wherein the data of the point symmetrical to the reflection means corresponding to the first specular equation data is regarded as first reflection point cloud data.
In a specific example of the embodiment of the present invention, the reflection device corresponding to the first specular equation data is a first reflection device. That is, a symmetric point of the first point cloud point behind the first reflecting device is generated, the point is recorded as the first reflecting point cloud point, and the corresponding data is first reflecting point cloud data.
And 106, performing position judgment processing according to the first reflection point cloud data and the preset region of interest data, and determining whether the first reflection point cloud point corresponding to the first reflection point cloud data is in the region of interest.
Specifically, the point corresponding to the first radiation point cloud data is a first reflection point cloud point, and whether the first reflection point cloud point is in the region of interest is determined, and the specific method is the same as that in step 102, and will not be described herein. When the first reflection point cloud point is within the region of interest, it is indicated that the first point cloud point is possibly three-dimensional point cloud data generated by capturing by the time-of-flight camera after receiving the light reflected by the object surface through the reflection device, and step 107 is performed. When the first reflection point cloud point is not in the region of interest, the first reflection point cloud point is obtained and generated by a time-of-flight camera after being reflected by the reflection device, and the first point cloud data is not processed. The processor processes the next first point cloud data in the original three-dimensional point cloud data in step 102 and the following steps.
And step 107, performing reflection transformation processing according to the first reflection point cloud data and the second mirror equation data to obtain second reflection point cloud data.
Specifically, in the embodiment of the present invention, the preset mirror equation data includes two sets of mirror equation data, wherein one set of mirror equation data is determined to be the first mirror equation data, and the remaining one set of mirror equation data is determined to be the second mirror equation data. And the processor calculates according to the three-dimensional coordinate value of the first reflection point cloud data and the second mirror equation data according to the plane mirror imaging principle, and generates data of points of the first reflection point cloud point and the reflection device corresponding to the second mirror equation data, wherein the points are symmetrical.
In a specific example of the embodiment of the present invention, the reflecting device corresponding to the second mirror equation is a second reflecting device. That is, a symmetrical point of the first reflection point cloud point behind the second reflection device is generated, the point is recorded as a second reflection point cloud point, and the corresponding data is second reflection point cloud data.
At this time, it cannot be determined whether the first reflection point cloud is on the object surface.
Specifically, the preset camera center position data is position data of a center point of the time-of-flight camera, which is determined after analysis and calculation after determining the positions of all components of the three-dimensional reconstruction system of the object, and has three-dimensional coordinate values.
In a preferred scheme of the embodiment of the invention, determining whether the first reflected point cloud data is reliable point cloud data according to the first point cloud data, the second reflected point cloud data and the preset camera center position data specifically comprises the following steps:
firstly, performing space distance calculation processing according to three-dimensional coordinate data of first point cloud data and preset camera center position data to obtain first distance data;
secondly, performing space distance calculation processing according to the three-dimensional coordinate data of the second reflection point cloud data and the preset camera center position data to obtain second distance data;
and then judging the sizes of the first distance data and the second distance data, and when the first distance data is smaller than the second distance data, indicating that the cloud point of the first reflection point is positioned on the surface of the object. At this time, the first reflection point cloud data is determined as reliable point cloud data. Otherwise, the first reflection point cloud point is not on the surface of the object. At this time, the first reflection point cloud data is determined as unreliable points
When the first reflection point cloud data is reliable point cloud data, step 109 is performed. When the first reflection point cloud point is not the reliable point cloud point, processing the next first point cloud data in step 102 and the following steps.
And step 109, storing the first reflection point cloud data in the three-dimensional point cloud data of the object.
And (3) after the first point cloud data in the original three-dimensional point cloud data are judged in the steps 102-109, the three-dimensional point cloud data of the object can be obtained. That is, the three-dimensional point cloud reconstruction of the object is completed. The point cloud data are obtained from first point cloud data in the original three-dimensional point cloud data, and the point cloud data are first reflection point cloud data obtained by carrying out reflection conversion processing on the first point cloud data once based on first mirror equation data.
In a preferred embodiment of the present invention, after generating the three-dimensional point cloud data of the object, the processor sends the three-dimensional point cloud data of the object to the display device. And the display equipment performs display output according to the three-dimensional point cloud data of the object.
According to the object three-dimensional reconstruction method and system provided by the embodiment of the invention, a frame of image of a target scene is acquired by using a time-of-flight camera, original three-dimensional point cloud data is generated, the acquired original three-dimensional point cloud data is combined with preset scene information data to analyze, reliable point cloud data in the original three-dimensional point cloud data is determined, the reliable point cloud data is generated through point cloud points in the original three-dimensional point cloud data, and finally, the object three-dimensional point cloud data is formed by a plurality of reliable point cloud data determined from the original three-dimensional point cloud and a plurality of reliable point cloud data generated according to the original three-dimensional point cloud data, so that the three-dimensional point cloud reconstruction of the object is completed. The method provided by the embodiment of the invention only needs to shoot one frame of image data of the target scene, has simple processing algorithm and small processing data volume, has a corresponding simple system structure, and can finish the reconstruction of the three-dimensional point cloud of the dynamic and static objects on the basis of reducing the equipment cost.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of function in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The foregoing detailed description of the invention has been presented for purposes of illustration and description, and it should be understood that the invention is not limited to the particular embodiments disclosed, but is intended to cover all modifications, equivalents, alternatives, and improvements within the spirit and principles of the invention.
Claims (9)
1. A method of three-dimensional reconstruction of an object, the method comprising:
the processor receives original three-dimensional point cloud data sent by the three-dimensional point cloud imaging device; wherein the original three-dimensional point cloud data comprises a plurality of first point cloud data;
performing position judgment processing according to the first point cloud data and preset region of interest data, and determining whether a first point cloud point corresponding to the first point cloud data is in a region of interest corresponding to the preset region of interest data;
when the first point cloud point is in the region of interest, determining the first point cloud data as reliable point cloud data, and storing the first point cloud data in object three-dimensional point cloud data;
when the first point cloud point is not in the region of interest, determining first mirror equation data according to the first point cloud data, preset mirror equation data and the preset region of interest data;
performing reflection transformation processing according to the first point cloud data and the first mirror equation data to obtain first reflection point cloud data;
position judgment processing is carried out according to the first reflection point cloud data and the preset region of interest data, and whether a first reflection point cloud point corresponding to the first reflection point cloud data is in the region of interest is determined;
when the first reflection point cloud point is in the region of interest, carrying out reflection transformation processing according to the first reflection point cloud data and the second mirror equation data to obtain second reflection point cloud data;
determining whether the first reflected point cloud data is reliable point cloud data according to the first point cloud data, the second reflected point cloud data and preset camera center position data;
when the first reflection point cloud data is reliable point cloud data, storing the first reflection point cloud data in the three-dimensional point cloud data of the object;
the method comprises the steps of carrying out position judgment processing according to first point cloud data and preset region of interest data, and determining whether first point cloud points corresponding to the first point cloud data are in a region of interest corresponding to the preset region of interest data specifically comprises:
according to the first point cloud data and the preset region of interest data, determining coordinate values of projection points of the first point cloud points on a plane of interest corresponding to the preset region of interest data;
determining whether the projection point is in an area of interest corresponding to the preset area of interest data by adopting a ray method;
when the projection point is in the region of interest, determining that the first cloud point is in the region of interest; and when the projection point is not in the region of interest, determining that the first cloud point is not in the region of interest.
2. The method according to claim 1, wherein the preset mirror equation data includes two sets of mirror equation data, and the determining the first mirror equation data according to the first point cloud data, the preset mirror equation data, and the preset region of interest data is specifically:
determining a group of mirror equation data in the preset mirror equation data as judging mirror equation data;
performing reflection transformation processing according to the first point cloud data and the determined mirror equation data to obtain reflected point cloud data;
performing position judgment processing according to the reflection point cloud data and the preset region of interest data, and determining whether the reflection point cloud point corresponding to the reflection point cloud data is in the region of interest;
and when the reflection point cloud point is in the region of interest, determining the determined mirror equation data as first mirror equation data, otherwise, determining the other set of mirror equation data in the preset mirror equation data as first mirror equation data.
3. The method of three-dimensional reconstruction of an object according to claim 1, wherein determining whether the first reflected point cloud data is reliable point cloud data according to the first point cloud data, the second reflected point cloud data, and preset camera center position data specifically comprises:
performing distance calculation processing according to the first point cloud data and preset camera center position data to obtain first distance data;
performing distance calculation processing according to the second reflection point cloud data and the preset camera center position data to obtain second distance data;
and when the first distance data is smaller than the second distance data, determining the first reflection point cloud data as reliable point cloud data.
4. The method of three-dimensional reconstruction of an object according to claim 1, wherein before the processor receives the raw three-dimensional point cloud data transmitted by the three-dimensional point cloud imaging device, the method further comprises:
the three-dimensional point cloud imaging device receives an externally input image acquisition instruction;
shooting a target scene according to the image acquisition instruction, and generating original three-dimensional point cloud data of the target scene;
the three-dimensional point cloud imaging device sends the original three-dimensional point cloud data to the processor.
5. The method of three-dimensional reconstruction of an object according to claim 4, wherein the three-dimensional point cloud imaging device is a time-of-flight camera.
6. The method of three-dimensional reconstruction of an object according to claim 1, further comprising:
the processor sends the three-dimensional point cloud data of the object to a display device;
and the display equipment displays and outputs according to the three-dimensional point cloud data of the object.
7. A three-dimensional reconstruction system for an object, the system comprising: the three-dimensional point cloud imaging device, the first reflecting device, the second reflecting device and the processor;
the three-dimensional point cloud imaging device is used for receiving an externally input image acquisition instruction, shooting a target scene according to the image acquisition instruction and generating original three-dimensional point cloud data of the target scene;
the first reflecting device is used for reflecting the light emitted by the three-dimensional point cloud imaging device to the surface of the object and secondarily reflecting the light reflected by the surface of the object to the three-dimensional point cloud imaging device;
the second reflecting device is used for reflecting the light emitted by the three-dimensional point cloud imaging device to the surface of the object and secondarily reflecting the light reflected by the surface of the object to the three-dimensional point cloud imaging device;
the processor is used for receiving the original three-dimensional point cloud data sent by the three-dimensional point cloud imaging device; wherein the original three-dimensional point cloud data comprises a plurality of first point cloud data;
the processor is further configured to perform position judgment according to the first point cloud data and preset region of interest data, and determine whether a first point cloud point corresponding to the first point cloud data is in a region of interest corresponding to the preset region of interest data;
when the first point cloud point is in the region of interest, the processor is further configured to determine the first point cloud data as reliable point cloud data, and store the first point cloud data in object three-dimensional point cloud data;
when the first point cloud point is not in the region of interest, the processor is further configured to determine first mirror equation data according to the first point cloud data, preset mirror equation data, and the preset region of interest data;
the processor is further used for carrying out reflection transformation processing according to the first point cloud data and the first mirror equation data to obtain first reflection point cloud data;
the processor is further configured to perform position judgment processing according to the first reflection point cloud data and the preset region of interest data, and determine whether a first reflection point cloud point corresponding to the first reflection point cloud data is in the region of interest;
when the first reflection point cloud point is in the region of interest, the processor is further configured to perform reflection transformation processing according to the first reflection point cloud data and the second mirror equation data to obtain second reflection point cloud data;
the processor is further configured to determine whether the first reflected point cloud data is reliable point cloud data according to the first point cloud data, the second reflected point cloud data, and preset camera center position data;
when the first reflection point cloud data is reliable point cloud data, the processor is further configured to store the first reflection point cloud data in the three-dimensional point cloud data of the object;
the method comprises the steps of carrying out position judgment processing according to first point cloud data and preset region of interest data, and determining whether first point cloud points corresponding to the first point cloud data are in a region of interest corresponding to the preset region of interest data specifically comprises:
according to the first point cloud data and the preset region of interest data, determining coordinate values of projection points of the first point cloud points on a plane of interest corresponding to the preset region of interest data;
determining whether the projection point is in an area of interest corresponding to the preset area of interest data by adopting a ray method;
when the projection point is in the region of interest, determining that the first cloud point is in the region of interest; and when the projection point is not in the region of interest, determining that the first cloud point is not in the region of interest.
8. The object three-dimensional reconstruction system according to claim 7, further comprising a display device:
the display device is used for receiving the three-dimensional point cloud data of the object sent by the processor;
the display device is also used for displaying and outputting according to the three-dimensional point cloud data of the object.
9. The object three-dimensional reconstruction system according to claim 8, further comprising:
the three-dimensional point cloud imaging device is in communication connection with the processor in a wired or wireless mode;
the processor is in communication connection with the display device through a wired or wireless communication mode.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010358037.7A CN111583392B (en) | 2020-04-29 | 2020-04-29 | Object three-dimensional reconstruction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010358037.7A CN111583392B (en) | 2020-04-29 | 2020-04-29 | Object three-dimensional reconstruction method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111583392A CN111583392A (en) | 2020-08-25 |
| CN111583392B true CN111583392B (en) | 2023-07-14 |
Family
ID=72111898
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010358037.7A Active CN111583392B (en) | 2020-04-29 | 2020-04-29 | Object three-dimensional reconstruction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111583392B (en) |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2887058B1 (en) * | 2005-06-10 | 2007-08-31 | Daniel Ackerman | METHOD AND DEVICE FOR 3D RECONSTRUCTION OF AN OBJECT FROM MULTIPLE 2D IMAGES |
| US8290305B2 (en) * | 2009-02-13 | 2012-10-16 | Harris Corporation | Registration of 3D point cloud data to 2D electro-optical image data |
| US20130182079A1 (en) * | 2012-01-17 | 2013-07-18 | Ocuspec | Motion capture using cross-sections of an object |
| US9619691B2 (en) * | 2014-03-07 | 2017-04-11 | University Of Southern California | Multi-view 3D object recognition from a point cloud and change detection |
| US20170135655A1 (en) * | 2014-08-08 | 2017-05-18 | Carestream Health, Inc. | Facial texture mapping to volume image |
| WO2017113260A1 (en) * | 2015-12-30 | 2017-07-06 | 中国科学院深圳先进技术研究院 | Three-dimensional point cloud model re-establishment method and apparatus |
| US11734892B2 (en) * | 2018-07-26 | 2023-08-22 | Shenzhen University | Methods for three-dimensional reconstruction of transparent object, computer devices and storage mediums |
| CN109685891B (en) * | 2018-12-28 | 2023-09-01 | 鸿视线科技(北京)有限公司 | Building three-dimensional modeling and virtual scene generation method and system based on depth image |
| CN110415342B (en) * | 2019-08-02 | 2023-04-18 | 深圳市唯特视科技有限公司 | Three-dimensional point cloud reconstruction device and method based on multi-fusion sensor |
| CN110490917A (en) * | 2019-08-12 | 2019-11-22 | 北京影谱科技股份有限公司 | Three-dimensional rebuilding method and device |
| CN110686600B (en) * | 2019-11-05 | 2021-08-06 | 北京深测科技有限公司 | Measuring method and system based on flight time measurement |
| CN110954017B (en) * | 2019-11-22 | 2021-03-16 | 武汉大学 | Method for acquiring and resolving laser scanning data reflected by any curved mirror |
-
2020
- 2020-04-29 CN CN202010358037.7A patent/CN111583392B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111583392A (en) | 2020-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11328446B2 (en) | Combining light-field data with active depth data for depth map generation | |
| JP6484729B2 (en) | Unmanned aircraft depth image acquisition method, acquisition device, and unmanned aircraft | |
| US20170059305A1 (en) | Active illumination for enhanced depth map generation | |
| EP3373251A1 (en) | Scan colorization with an uncalibrated camera | |
| CN101952855B (en) | Method and camera for the real-time acquisition of visual information from three-dimensional scenes | |
| US20140037194A1 (en) | Three-dimensional point cloud position data processing device, three-dimensional point cloud position data processing system, and three-dimensional point cloud position data processing method and program | |
| CN102763420B (en) | depth camera compatibility | |
| US20130335535A1 (en) | Digital 3d camera using periodic illumination | |
| JP4374625B2 (en) | Image generating apparatus and method | |
| WO2021140886A1 (en) | Three-dimensional model generation method, information processing device, and program | |
| CN111383285B (en) | Sensor fusion calibration method and system based on millimeter wave radar and camera | |
| US20070247612A1 (en) | System and method for improving lidar data fidelity using pixel-aligned lidar/electro-optic data | |
| WO2005043076A1 (en) | Method for calibrating a camera-laser-unit in respect to a calibration-object | |
| CN111047650B (en) | Parameter calibration method for time-of-flight camera | |
| JP2003130621A (en) | Method and system for measuring three-dimensional shape | |
| CN108286945A (en) | The 3 D scanning system and method for view-based access control model feedback | |
| CN113160328A (en) | External reference calibration method, system, robot and storage medium | |
| CN108332660B (en) | Robot three-dimensional scanning system and scanning method | |
| CN107869968A (en) | A kind of quick three-dimensional scan method and system suitable for complex object surface | |
| JP4843544B2 (en) | 3D image correction method and apparatus | |
| KR20220012375A (en) | Apparatus and method for providing around view | |
| CN107564051B (en) | Depth information acquisition method and system | |
| CN115908720A (en) | Three-dimensional reconstruction method, device, equipment and storage medium | |
| JP2010176325A (en) | Device and method for generating optional viewpoint image | |
| CN111583392B (en) | Object three-dimensional reconstruction method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |