KR20030015625A - Calibration-free Approach to 3D Reconstruction Using A Cube Frame - Google Patents

Abstract

PURPOSE: A method for a three-dimensional reconstruction of an object without camera calibration using a cube frame is provided to model the object regardless of camera parameter and position of a light source. CONSTITUTION: A method for restoring a three-dimensional structure of an object includes the steps of taking a photograph of a scene that a line type light emitted from a light plane projector(5) illuminate a target object(4) inside a cube frame(3), recognizing a plurality of LEDs(1) attached to the cube frame(3) in the image and setting up coordinates based on the recognized position, calculating the coordinates of places where a light plane meets the edges of the cube frame(3), and calculating the coordinates of a light stripe formed by making the target object meet with the light plane.

Description

CALIBRATION-free Approach to 3D Reconstruction Using A Cube Frame}

The present invention relates to the reconstruction of a three-dimensional structure of a target object, and a device that performs this function is called a three-dimensional scanner. 3D scanners can be divided into passive and active methods. The passive method is easy to use because it uses clues such as parallelism, shading, parallax, etc. that appear in the image, and is inexpensive. On the other hand, the active method has an advantage of obtaining accurate 3D structural information, but has a disadvantage of being expensive because of using a special apparatus.

In the passive method, the amount of information that can be extracted is small because the modeling is performed using visual cues that can perceive the depth of one or several images. To make up for the drawbacks, most of them use texture mapping to make them look visually plausible. Therefore, when more accurate and dense modeling is needed, the active method is used. Active methods provide information in the form of range images through modeling. A range image is a special form of digital image where each pixel represents the distance from the reference frame to the visible portion of the scene. To obtain range images, active methods use optical range sensors such as radio detectors, noir interferometry, focusing and trigonometry.

Commercial systems using active methods include Cyberware's Head & Face Color 3DScanner, Eyetronics' ShapeSnatcher, and Real3D's RealScan. However, these devices are expensive and bulky, making them difficult to use universally.

Bouguet, on the other hand, suggested using simple tools such as cameras, desk lamps and pencils. This method calculates the time included in the shadow of a pencil for every pixel of the image, and calculates the point where the shadow plane formed by the lamp and the shadow meets the point where a straight line projected back to the pixel in the COP meets a range image. . However, since the information cannot be extracted from the part that is not shown in the image and the part that is shadowed by the object, the scanning of the camera and the lamp must be performed several times. Therefore, the position must be corrected every time the camera or light source is moved, and all range images must be matched into one image.

Therefore, a technical object of the present invention is to make it possible to obtain precise information while using a simple device in generating a three-dimensional structure of an object.

Specifically, by using a cube-shaped frame to provide a method of modeling the three-dimensional structure of the object while moving freely without correcting the position of the optical plane projector and the camera used as a light source, while using a device composed of simple tools It is possible to obtain accurate three-dimensional structural information while increasing the convenience of. Also, in order not to add a separate matching process, each scanned information is shown to indicate a relative position with respect to one reference coordinate system.

1 is a flow chart showing a process of using the three-dimensional scanner system to which the present invention is applied.

2 is an overall structural diagram of a three-dimensional scanner system according to an embodiment of the present invention.

3 is a flowchart illustrating the image processing process illustrated in FIG. 1 by subdividing into more specific steps.

4 is a diagram illustrating an example of an input image captured by a camera.

5 is a diagram illustrating only LED information in an input image.

6 is a diagram showing the relationship between the LDP and the light plane in the cube frame.

7 is a diagram showing an example of a vanishing point of a perspective projected cube.

8 is a diagram showing a correlation between an LDP and a vanishing point on an optical plane.

9 is a diagram showing an example of progressive calculation of three-dimensional coordinates on the optical plane.

10 shows an example of a Voronoi diagram and a dealer's triangulation.

11 is a diagram illustrating an example of a result of performing a surface reconstruction step.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram schematically illustrating a process of obtaining a result using a three-dimensional scanner to which the present invention is applied. First, an image of a target object is obtained from a camera (10), and a three-dimensional structure of the object is obtained using the result (20), and finally, a surface reconstruction step (30) is generally used in a computer system. Obtain a model where the surface is composed of possible triangle meshes.

2 illustrates the overall system of the present invention. The system proposed in the present invention is composed of a cube frame (3) with 10 LEDs (1), a digital camera (6), a light plane projector (5), and an image processing computer (7). do. One vertex (1) is attached to each vertex of the cube frame (3), and two more auxiliary LEDs (2) are attached to indicate the z-axis and the origin of the coordinate system. One auxiliary LED 2 is attached to the inside and the outside of the cube corners so that at least one is always captured in the image regardless of the position of the camera 6. The image obtained from the camera 6 is input to the computer system 7 to obtain a surface model of the desired object 4 through image processing.

FIG. 3 is a flowchart illustrating the general process of the image processing described in FIG. 1 in more detail.

Acquiring the three-dimensional structure of the object 20 is a step of restoring the three-dimensional structure of the object by photographing the target object and analyzing vanishing point information displayed on the image by using a simple tool without restoring camera parameters. 10, the LED identification and coordinate setting 21, the three-dimensional coordinate calculation 22 of the LDP, the thinning 23 of the light streaks, and the three-dimensional coordinate calculation 24 of the object.

The image obtainable in the image acquisition step 10 is composed of points and lines as shown in FIG. When acquiring an image, the light emitted from the light plane projector 5 always meets at three parallel edges of the cube frame 3. In this case, since the plane defined by three points is the same plane as the plane emitted from the projector, these three points are called LDP (Light plane Definition Points) 100, and the curve where the light plane meets the object is a light stripe. It is referred to as (110). If the optical plane projector and the camera are directed in parallel directions, the light streaks appear in a straight line, so that the camera 6 is located away from the light plane projector 5 as shown in FIG. Should be As a result, since no other light source is used in the acquired image, only the light streaks 110, the LDP 100, and the LED 1 appear.

In the LED identification and coordinate setting step 21, the LED 1 is selected from the acquired image and the coordinates are set. The bright part in the acquired image includes the LDP 100 and the light streaks 110 in addition to the LED 1. In order to distinguish them, they are first classified into lines and dots and extracted in the order of the points on the LED 1, the LDP 100, and the light streaks 110. Bright pixels are divided into connected component units to separate bright parts of an image into dots and lines. The points and lines are distinguished by calculating the center of each connected component and finding the variance of the distance from each pixel of the connected component to the center. This is based on the feature that the line component has a large dispersion of the distance between the center and the pixels.

The process of identifying and extracting the LED 1 will be described with reference to FIG. 5. Finding the convex hull of the points in the image, the point on the convex hull To Can be identified by the LED. point To After finding the , internal points from them If you find it, you will find all LEDs. Is Wow Vanishing point by and Wow Vanishing point, Wow Between each vanishing point by , , Search by finding the intersection of the three straight lines passing through. Extracting these seven LEDs can restore the cube's shape.

In addition to the LEDs 1 attached to each vertex, there are auxiliary LEDs 2 for indicating the z-axis of the coordinate system. Once you find the z axis, you can determine the other two axes using the right-hand screw law. After determining the coordinate axes, the three-dimensional coordinates of each vertex are normalized and assigned. Therefore, since the points on the object surface in the cube frame all have coordinates of 0 to 1, if only the length of one side of the actual cube frame is known, the coordinates of the actual size can be obtained.

In the three-dimensional coordinate calculation step 22 of the LDP, the three-dimensional coordinates of the LDP are calculated using the properties of the vanishing point and the intersection rate. FIG. 6 is a diagram showing a relationship between the cube frame 3 and the light plane 120 including the LDP 100. As shown in FIG. 2, the light emitted from the optical plane projector 5 meets the corner of the cube frame 3 at three LDPs 100, and the three planes define the optical plane 120. Dots on light streaks 110 Is three points as shown in Figure 6 , , Above the light plane 120, which is defined as.

Fig. 7 shows vanishing points and vanishing lines of the cubes perspectively projected. The number of vanishing points in the image depends on the position of the cube and the camera. FIG. 7A shows one vanishing point, FIG. 7B shows two vanishing points, and FIG. 7C shows three vanishing points. In the case of FIGS. 7A and 7B where two vanishing points are two or less, the vanishing points which do not appear in the image are at infinity. The vanishing line for a plane is the collection of vanishing points where all parallel straight lines parallel to that plane meet. In the present invention, in order to calculate the three-dimensional coordinates of the LPD, a vanishing point where the parallel edges of the cube frame are projected and used, and a cross-ratio having an invariant property in projective geometry are used.

The crossover rate of Figure 8 , , , As shown in Equation 1, four points on the same straight line are defined as follows.

At this time, In the video and Indicates the distance of. The ratio of Equation 1 not only remains constant regardless of the camera position, but also in the Euclidean geometry, which represents the real world. Thus, in the real world For Equation 1 is used as is to calculate the ratio of.

In equation 2 In three dimensions Wow Indicates the distance of. And, Is an infinite point, so the exact coordinates are unknown, so the distance between the two points is unknown It can be seen. Thus, in Equation 2, the distance ratio in three dimensions can be known using only the crossover ratio in the image. Using this The coordinate of can be calculated as Equation 3. Remainder Wow Vanishing point about The four points on the straight line passing through can be used to calculate three-dimensional coordinates in the same way. In the case where the vanishing point is at infinity, that is, in the case of FIGS. 7A and 7B, the same method may be used. In equation 2 Since the ratio of the distance in the three-dimensional image is equal to the ratio of the distance in the image. Therefore, the coordinates of the LDP can be calculated more simply than in the case of three vanishing points.

The thinning step 23 of the light streaks is a thinning process as a preprocessing step for calculating the coordinates of the points in the light streaks 110. Through this process, the light streaks are represented by a single line having no thickness.

In the final process of obtaining the three-dimensional structure of the object 20, the three-dimensional coordinate calculation step 24 of the object calculates the coordinates of the points on the thinned light stripes using the coordinates of the vertices of the cube frame and the LDP. One point on the light streaks as shown in FIG. Is in the light plane passing through the three LDPs. point Are two straight lines in space and Is the intersection of. At this time, silver Parallel to Is a straight line through silver Parallel to Straight through. Two parallel straight lines in space Wow Is no longer parallel and vanishes when perspective is projected onto the image plane Will meet at Wow Degree Meet in. At this time, Is contained in the horizontal plane Wow Vanishing point Is shown in Figure 9 and Is included in the vanishing line passing through Is and It will be on the vanishing line passing through.

point Three-dimensional coordinates of Computation basically uses interpolation using cross rate. In Figure 9 To calculate Rate for Four Points on Pinterest and Use However, the information so far known is the four vertices of the cube frame, the two-dimensional, three-dimensional coordinates of the LDP, and the two-dimensional coordinates of the vanishing point. Therefore, based on the information so far The remaining necessary three-dimensional coordinates are computed through incremental computation. To calculate and You need to know and , and Can be calculated using.

By performing the three-dimensional structure acquisition step 20 of the object described so far, the three-dimensional coordinates on the surface of the object can be easily obtained by using a simple equation using the vanishing point and the intersection rate of the cube frame captured in the image regardless of the position of the camera and the light source. Can be. These calculated points on the surface of the object are point clouds of information that have no information other than three-dimensional coordinates and cannot be used in practical applications in computer graphics. We now describe a surface reconstruction step 30 that organizes these points and reconstructs the surface of an object into a triangular network structure for use in applications.

Surface reconstruction step 30 is a step of reconstructing the model of the object consisting of a network of triangles from the points calculated in the three-dimensional structure acquisition step 20 of the object, is obtained through a triangle extraction algorithm. FIG. 10A shows a set of sample points which are inputs of the surface reconstruction step, and FIG. 10B shows an object model composed of a triangular network which is the result of a triangle extraction algorithm.

The surface reconstruction step 30 is composed of a process of triangulation extraction algorithm 31, the surface information acquisition 32 of the object as shown in FIG. After constructing a network of basic triangles of the target object by applying a triangle extraction algorithm (31), such as the dealer's triangulation algorithm, and finally removing the incorrectly created triangles using the normal vectors, the topology and isomorphism A network of triangles is created that are homomorphic. The dealer's triangular partitioning algorithm presented as an example of the triangle extraction algorithm in the present invention uses a randomized incremental algorithm and has a robust characteristic because it performs an exact integer operation. However, the present invention is not limited to the use of the dealer's triangulation algorithm as the triangle extraction algorithm, but includes the use of any triangle extraction algorithm for surface reconstruction.

11 shows an example of the results of the surface reconstruction in the surface reconstruction step 30.

Although the present invention has been described with reference to the most practical and preferred embodiments, the present invention is not limited to the above disclosed embodiments, but also includes various modifications and equivalents within the scope of the following claims.

According to the embodiments described above, three-dimensional modeling of an object may be obtained using simple tools such as a cube frame, an optical plane projector, and a camera.

Unlike the existing 3D scanner system, which requires precise correction of camera parameters and strict position of the camera and light source, the system applying the present invention does not require correction of camera parameters and has the advantage of freely moving the light source and the camera. . In the system to which the present invention is applied, an object to be modeled is placed inside a cube frame. Since the frame is accurately identified in the acquired image, and the vanishing point of the parallel edge of the frame is used to calculate the position of an arbitrary point on the object surface, the object can be modeled regardless of the position of the camera parameter and the light source. Also, since the calculated 3D coordinates are relative to the frame, they are represented as a set of points without matching. Finally, the limited system reconstructs the surface of an object into a triangular network structure using a triangle extraction algorithm, making it applicable to computer graphics applications including virtual reality.

The present invention can be applied to applications such as an internet shopping mall, a virtual reality application, a three-dimensional game, and the like that require three-dimensional modeling information.

Claims (6)

Objects include a cube frame with LEDs attached to each vertex, an optical plane projector that emits light in the form of a line, a camera for image acquisition, and an image processing computer that analyzes the captured image to create a triangular mesh object model. 3D structural restoration system. The method of claim 1, The cube frame One LED is attached to each vertex, and an auxiliary LED is attached to indicate the z-axis and the origin of the coordinate system, so that the camera's position can be automatically extracted wherever the camera's point of view is located. system. 3D structure restoration method comprising the step of acquiring the three-dimensional structure of the object to calculate the three-dimensional coordinates of the photographed object by photographing the target object and the surface reconstruction step to obtain a three-dimensional network model by applying a triangle extraction algorithm . The method of claim 3, The three-dimensional structure acquisition step of the object An image acquisition step of photographing a scene in which a line-shaped light emitted from an optical plane projector illuminates an object inside a cube frame, and recognizes the LED attached to the cube frame in the captured image and sets coordinates based on the recognized position. LED identification and coordinate setting step, LDP's three-dimensional coordinate calculation step that calculates the coordinates where the optical plane meets the edge of the cube frame, and three-dimensional object's coordinate that calculates the coordinates of the light streaks formed by the meeting of the object and the optical plane A method for restoring a three-dimensional structure of an object, including the step of calculating a coordinate. The method of claim 4, wherein The three-dimensional coordinate calculation step of the LDP and the three-dimensional coordinate calculation step of the object A method for restoring a three-dimensional structure of an object, characterized by using a vanishing point where the parallel edges of a cube frame are projected to meet three-dimensional coordinates and an intersection ratio having an invariant property in the projection geometry. The method of claim 3, Surface reconstruction step A method of restoring a three-dimensional structure of an object comprising applying a triangle extraction algorithm to a set of points on an object surface and removing a triangle that was incorrectly created using a normal vector.