KR20180074976A - Three dimension scanner using multi camera - Google Patents

Abstract

The present invention relates to a three-dimensional scanner using a multi-camera, capable of reducing scanning distortion occurring during three-dimensional scanning. According to the present invention, the three-dimensional scanner using the multi-camera includes: a housing unit; a chamber unit disposed inside the housing unit and including a base part and a plurality of side parts to allow a scanning target object to be positioned inside a scanning space defined by the base part and the side parts; a plurality of image sensor units installed on the side parts to photograph the scanning target object positioned in the chamber unit; an image processing unit for processing image information sensed by the image sensor units to generate a three-dimensional image; and a control unit for controlling the image sensor units and the image processing unit, wherein each of the image sensor units is composed of at least two multi-cameras, and the control unit controls the image sensor unit, which has sensed image information that does not satisfy a predetermined condition among the image information photographed by the image sensor units, to perform photographing again.

Description

[0001] THREE DIMENSION SCANNER USING MULTI CAMERA [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional scanner using a multi-camera, and more particularly, to a three-dimensional scanner using a multi-camera, which generates three-dimensional image rendering data in which noise is reduced by capturing image information without shaking by a plurality of multi- Dimensional scanner.

BACKGROUND ART [0002] In recent years, a technique of outputting three-dimensional sculptures (hereinafter, referred to as '3D printing') has been popular. With 3D printing technology, it is possible to produce custom products without going through the mold manufacturing process.

For 3D printing, 3D rendering is required. For 3D rendering, methods of direct drafting using 3D drawing program and 3D modeling by taking a product using camera are widely used.

In particular, a method of 3D modeling using a camera is generally performed by using two or more cameras in order to convert 2D photographs into 3D modeling. 3D modeling using a camera is generally performed by scanning the periphery of a target product using a scanner.

However, when the object is scanned using the 3D scanning, there is a problem that the 3D modeling is not completed when the scanning object moves, or the original 3D shape is distorted by including the noise in the 3D modeling file.

In order to solve such a problem, a method of scanning a target object with a 3D scanner while rotating the object on a rotary plate has been proposed. However, when an object moves during rotation, scanning distortion may also occur.

Therefore, there is a need for a new type of 3D scanner technology for 3D scanning of an object that is expected to move.

According to the present invention, which is devised in view of the above-described necessity, image information captured in a state in which shaking occurs due to movement of an object or movement of a scanner during scanning is automatically eliminated, thereby reducing scanning distortion occurring during 3D scanning Dimensional scanner using a multi-camera.

In order to achieve the above object, a three-dimensional scanner using a multi-camera according to the present invention comprises a housing part, a scanning part located on the inside of the housing part and including a base part and a plurality of side parts, A plurality of image sensor units provided on the plurality of side surface portions for imaging the object located in the chamber portion, and a control unit for processing the image information sensed by the plurality of image sensor units And a control unit for controlling the plurality of image sensor units and the image processing unit, wherein each of the plurality of image sensor units is constituted by at least two or more cameras, And a plurality of image sensor units The image sensor unit which has not met the condition of the image sensor unit is imaged again.

In this case, the plurality of image sensor units may include a first sensor module including a first camera and a second camera provided on a first side surface portion of the plurality of side surface portions, a third camera provided on a second side surface portion of the plurality of side surface portions, A third sensor module composed of a fourth camera and a fifth camera installed on a third side portion of the plurality of side portions, and a third sensor module composed of a seventh camera and an eighth camera provided on the fourth side portion of the plurality of side portions, And a fourth sensor module composed of a first sensor module and a second sensor module.

In this case, the control unit controls the first sensor module, the second sensor module, the third sensor module, and the fourth sensor module to sequentially capture the object at least three times or more.

In this case, the image processing unit may process one image information for each sensor module among the first to fourth plurality of image information continuously picked up by the first to fourth sensor modules according to predefined conditions And transmits the information on each sensor module and the selected image information to the control unit as first to fourth sensing information,

The control unit controls to again pick up the sensor module which has not picked up the image information of the selected one of the first to fourth sensor modules based on the first to fourth sensing information.

In this case, the image processing unit may compare the first image information and the second image information among the plurality of image information, compare the second image information and the third image information, And compares the first image information to select image information based on the pixel difference value.

According to various embodiments of the present invention, image information captured in a state in which shake occurs due to movement of a target object or movement of a scanner during scanning is automatically removed and image sensing is performed. Thus, scanning distortion occurring during 3D scanning The effect is obtained.

1 is a block diagram illustrating a three-dimensional scanner using a multi-camera according to an embodiment of the present invention,
2 is a block diagram illustrating an image sensor unit of a three-dimensional scanner using a multi-camera according to an embodiment of the present invention.
3 is a view for explaining an image picked up by the image sensor unit according to an embodiment of the present invention,
4 is a view for explaining a process of comparing image information constituting the first image information shown in FIG. 3,
5 is a block diagram illustrating an image processing unit of a three-dimensional scanner using a multi-camera according to an embodiment of the present invention.
FIG. 6 is a block diagram illustrating a control unit of a three-dimensional scanner using a multi-camera according to an embodiment of the present invention,
FIG. 7 is a flowchart illustrating a three-dimensional scanning operation of a three-dimensional scanner using a multi-camera according to an embodiment of the present invention.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the embodiments. Further, the accumulation or ratio of the drawings in the various embodiments of the present invention is merely exemplary, and the elements of the present invention shown in the drawings according to the embodiments may be designed differently in shape, form, size, or size.

1 is a block diagram illustrating a three-dimensional scanner using a multi-camera according to an embodiment of the present invention. Referring to FIG. 1, a three-dimensional scanner 100 using a multi-camera according to an embodiment of the present invention includes a housing unit 110, a chamber unit 120, an image sensor unit 130, an image processing unit 140, And a control unit 150. Since the 3D scanner according to the embodiment of the present invention is an electronic device, it may include a power supply unit (not shown) and a user interface (not shown), but a detailed description thereof will be omitted.

The housing part 110 may have a rectangular parallelepiped shape. The housing part 110 may be made of plastic, metal, synthetic resin, or a combination thereof. The housing part 110 includes an inner space for constituting the inner chamber part 120. The housing unit 110 may include an image processing unit 140, a control unit 150, a power supply unit, and a user interface unit except for the image sensor unit 130. The housing part 110 may be designed in various forms, but the shape of the housing part 110 is not a core technical idea of the present invention, and a detailed description thereof will be omitted.

The chamber part 120 is located inside the housing part 110. The chamber portion 120 is composed of a base portion and a plurality of side portions. The base portion is designed on the bottom surface of the housing portion 110 and can be fixedly or pivotally attached to the housing portion 110. The plurality of side portions may be formed of four, five, or more depending on the shape of the housing portion 110. The chamber part 120 forms a scanning space by the base part and the four side parts, and the object to be scanned can be positioned in the scanning space.

According to one embodiment of the present invention, it is assumed that the plurality of side portions are four. The plurality of side portions may include a first side portion, a second side portion, a third side portion, and a fourth side portion. The detailed configuration will be described later with a separate drawing.

The plurality of image sensor units 130 are constituted by a device for capturing an image of a subject and converting it into an electrical signal. The image sensor may be composed of a camera including a CMOS or the like. Each camera of the plurality of image sensor units 130 is preferably composed of at least two or more cameras. The plurality of image sensor units 130 may be provided in a plurality of side portions, and may be composed of three, four, five, or more, depending on the number of the plurality of side portions. The plurality of image sensor units 130 can acquire image information by simultaneously capturing an object located inside the chamber 120.

The image processing unit 140 processes the electrical image information generated by the plurality of image sensor units 130 to obtain three-dimensional modeling information or filters the distorted image information based on the image information.

The control unit 150 controls various operations of the image sensor unit 130 and the image processing unit 140 and controls the overall operation of the three-dimensional scanner according to an exemplary embodiment of the present invention. That is, the control unit 150 controls the image sensor unit 130 to sense the image information that has not yet satisfied the predefined condition among the image information captured by the plurality of image sensor units 130 .

2 is a block diagram illustrating an image sensor unit of a three-dimensional scanner using a multi-camera according to an embodiment of the present invention. 2, the plurality of image sensor units 130 includes a first sensor module 131, a second sensor module 132, a third sensor module 133, and a fourth sensor module 134.

The first sensor module 131 includes a first camera 131a and a second camera 131b. The first camera 131a captures the first image IMG 1 and the second camera 131b captures the first image IMG 1. The second image IMG2 is picked up by the second image pick-up device.

The second sensor module 132 includes a third camera 132a and a fourth camera 132b. The third sensor 132b captures the third image IMG3 by the third camera 132a. The fourth sensor 133b, The fourth image IMG 4 is imaged by the second image processing unit.

The third sensor module 133 includes a fifth camera 133a and a sixth camera 133b. The fifth camera 133a captures a fifth image IMG 5, and the sixth camera 133b, The sixth image IMG 6 is picked up.

The fourth sensor module 134 includes a seventh camera 134a and an eighth camera 134b. The seventh camera 134a images the seventh image IMG 7 and the eighth camera 134b, (IMG 8) is imaged by the imaging device (not shown).

The plurality of image sensor units 130 according to an embodiment of the present invention can be configured to include a plurality of image sensors in the form of a multi-camera so that the depth sense of the subject can be calculated using the images captured by the two cameras, Images can be obtained.

In addition, since the plurality of image sensor units 130 include a multi-camera in four directions, it is possible to capture a subject in four directions, and synthesize the captured image information to perform three-dimensional modeling.

However, when a part of the images captured by the plurality of image sensor units 130 are distorted due to excessive light, insufficient light amount, or excessive light, the control unit 150 performs re- It is possible to prevent a distortion phenomenon from occurring in an image picked up in four directions.

In order to prevent distortion in the captured image information, the image processing unit 140 selects image information according to a predetermined reference among the received image information, and transmits the selected image information together with the image sensor sub information to the controller 150. Hereinafter, a process of selecting image information to be operated in the image processing unit 140 will be described with reference to FIG.

3 is a view illustrating an image captured by the image sensor unit according to an embodiment of the present invention. Referring to FIG. 3, first image information IMG 1 is captured by the first camera 131a. At this time, the first camera 131a captures an image three or more times for a predetermined time, so that the first camera 131a acquires the first image IMG 1-1, the second image IMG 1-2, and the third image IMG1- 3).

Similarly, the second camera 131b to the eighth camera 134b respectively capture three images, so that the plurality of image sensor units 130 acquire a total of 24 pieces of image information during a predetermined time. The three pieces of image information captured by the first camera 131a are referred to as a first image information set.

The image processing unit 140 selects only one piece of image information according to predetermined conditions in the image information set. If there is no image information satisfying the predefined condition, the image information captured by the image sensor unit is not selected. A specific method of selecting image information will be described in detail below.

4 is a view for explaining a process of comparing image information constituting the first image information set shown in FIG. Referring to FIG. 4, three images are captured during a predefined time. The first image information IMG 1-1 is imaged at the first time t1 and the second image information IMG 1-2 is imaged at the second time t2 and the third image information IMG 1-2 is imaged at the third time t3. (IMG 1-3). The first to third times t1 to t3 are defined as one imaging time, and the first camera 131a to the eighth camera 134b simultaneously capture a subject in one imaging time.

The set of image information captured during one imaging time may be composed of three pieces of image information. The image processing unit 140 includes a first sub-frame (Sub1) and a second sub-frame (Sub1) of the first image information (IMG 1-1), the second image information Sub 2) and the third sub-frame (Sub 3).

Specifically, the image processing unit 140 compares the pixel information of the first sub-frame (Sub 1) of the first image information (IMG 1-1) with the pixel information of the second sub-frame (Sub 2) of the second image information (IMG 1-2) do. The image processing unit 140 compares the pixel information of the second sub-frame Sub 2 of the second image information IMG 1-2 with the pixel information of the third sub-frame Sub 3 of the third image information IMG 1-3. The image processing unit 140 compares the pixel information of the first sub-frame Sub 1 of the first image information IMG 1-1 with the pixel information of the third sub-frame Sub 3 of the third image information IMG 1-3.

The image processing unit 140 is configured to process all the subframes constituting the first image information set IMG 1-1, IMG 1-2 and IMG 1-3, The pixel information is compared with each other.

The image processing unit 140 receives the pixel information of the pixel PX 1 located at the Nth column, the M row of the first sub-frame of the first image information IMG 1-1 and the pixel information of the pixel PX 1 located at the Mth row of the first image information IMG 1-2 The pixel information of the pixel PX2 located at the Nth column of the subframe and the pixel PX2 located at the Mth row are compared with each other to judge that the pixels are different if the brightness value or the NGB value, 1) and the second image information (IMG 1-2) by +1.

The image processing unit 140 compares all pixels of the first image information IMG 1-1 with all pixels of the second image information IMG 1-2 to calculate a difference value. In this manner, the image processing unit 140 calculates the total pixel difference value P_1 between the first image information IMG 1-1 and the second image information IMG 1-2, and outputs the second image information IMG 1-2, (P_2) between the third image information (IMG1 - 3) and the third image information (IMG1 - 3), and calculates the total pixel difference value ).

First image information Second image information Third image information First image information - P_1 (120) P_3 (100) Second image information P_1 (120) - P_2 (20) Third image information P_3 (100) P_2 (20) -

When the image processing unit 140 compares the first image information IMG 1-1 to the third image information IMG 1-3 to generate a pixel difference value table as shown in Table 1, And calculates the arithmetic mean of the difference values P_1 to P_3. The arithmetic mean is (120 + 20 + 100) / 3 = 80.

The image processing unit 140 judges that there is a shake or an excessive amount of light or a distortion of a light amount between the first and second image information having a pixel difference value larger than the arithmetic mean. Also, the image processing unit 140 judges that there is a shake or an excessive amount of light or a small amount of light between the first and third image information having a pixel difference value larger than the arithmetic average. On the other hand, the image processing unit 140 determines that there is no blurring or excessive amount of light, or light amount, color distortion, or the like between the second and third image information having pixel difference values smaller than the arithmetic mean. When the second and third image information are compared with the first image information, the image processing unit 140 determines that the first image information and the pixel difference value, which are judged as the distorted image information, are 120 and P_3, The second larger image information is finally selected as the image information.

First image information Second image information Third image information First image information - P_1 (120) P_3 (100) Second image information P_1 (120) - P_2 (110) Third image information P_3 (100) P_2 (110) -

When the image processing unit 140 compares the first image information IMG 1-1 to the third image information IMG 1-3 to generate a pixel difference value table as shown in Table 1, And calculates the arithmetic mean of the difference values P_1 to P_3. The arithmetic mean is (120 + 110 + 100) / 3 = 110.

The image processing unit 140 judges that there is a shake, an excessive amount of light, or a distortion of light amount between the first and second image information having a pixel difference value larger than the arithmetic mean (110). Also, the image processing unit 140 judges that there is a shake or excessive amount of light or a small amount of light between the second and third image information having a pixel difference value such as arithmetic mean (110). On the other hand, the image processing unit 140 determines that there is no shaking or excessive amount of light, light amount, color distortion, or the like between the first and third image information having pixel difference values smaller than the arithmetic mean.

However, since the pixel difference value P_3 between the first and third image information exceeds 50% of the arithmetic mean, the image processing unit 140 determines that distortion occurs between the first and third image information, Do not choose.

5 is a block diagram illustrating an image processing unit of a three-dimensional scanner using a multi-camera according to an embodiment of the present invention. Referring to FIG. 5, the image processing unit 140 receives four selected image information S_IMG 1 to S_IMG 4 from the four image sensor units 131 to 134. The image processing unit 140 generates four pieces of sensing information by matching each image sensor information with the selected image information, and transmits the sensing information to the control unit 150.

The control unit 150 receives the four pieces of sensing information Cap 1 to Cap 4 and parses the four pieces of sensing information to detect an image sensor unit that does not include image information. The control unit 150 extracts four pieces of image information from the four pieces of image information and outputs a control signal Ctl 5 to the image processing unit 140 to synthesize the extracted image information send.

The control unit 150 transmits a plurality of control signals Ctl 1 to Ctl 4 to an image sensor unit that does not include image information among the four pieces of sensing information Cap 1 to Cap 4. If there is no image information captured by the second image sensor unit, the control signal Ctl 2 is transmitted to the second image sensor unit.

6 is a block diagram illustrating a controller of a 3D scanner using a multi-camera according to an embodiment of the present invention. 6, the control unit 150 includes an input / output interface module 151, a determination module 152, and a storage module 153.

The input / output interface module 151 receives the four pieces of sensing information Cap 1 to Cap 4 from the image processing unit 140 or receives a plurality of control signals Ctl 1 to C 4 from the image sensor unit 130, To Ctl 5).

The determination module 152 parses the image sensor sub information and the image information among the four pieces of captured image information Cap 1 through Cap 4, and determines the type of the image sensor unit having no parsed image information. The determination module 152 determines that a distortion phenomenon has occurred during image capturing for an image sensor unit that has not generated image information and generates at least one control signal among the control signals Ctl 1 to Ctl 4.

The storage module 153 temporarily stores the four pieces of sensing information Cap 1 to Cap 4 and receives the four pieces of sensing information from the image processing unit 140. The four pieces of sensing information Cap 1 to Cap 4 ) To new four pieces of sensing information (Cap 1 to Cap 4) and stores them.

FIG. 7 is a flowchart illustrating a three-dimensional scanning operation of a three-dimensional scanner using a multi-camera according to an embodiment of the present invention. Referring to FIG. 7, a subject for three-dimensional scanning is placed in a chamber portion provided inside a three-dimensional scanner.

Step S710 is a step of detecting whether an object is positioned in the chamber part 120. [ In step S710, it is automatically detected by using an operation sensor or an illumination sensor. However, if the user inputs an object into the chamber and executes a command to start three-dimensional scanning using a switch or a button, step S710 may be omitted.

In step S730, when an imaging start command (also referred to as a three-dimensional scanning command) is inputted, the object located in the chamber unit 120 is simultaneously captured three or more times by four cameras at the same time. In step S730, all four cameras are preferably configured as dual cameras, and in some embodiments, they may be configured as tree pool cameras. A total of 24 pieces of image information can be obtained when an object is imaged by four dual cameras in step S730 and three images are continuously captured.

In step S750, pixel values of a plurality of captured image information are compared, and distortion-free image information is finally selected for each of the four dual cameras. The selected image information may be composed of a total of 8 image files.

If it is determined in step S750 that there is image information with distortion (Yes in step S750), the image sensor unit that has captured the distorted image information is instructed to re-capture the image (S770).

If it is confirmed in step S750 that no distorted image information is present (No in step S750), three-dimensional modeling of the object is performed using a total of eight pieces of image information (S790).

The basic principle is to acquire two images by using a dual camera for one subject, acquire depth values, and then acquire three-dimensional images of the two images . By repeating the same procedure for four directions, three-dimensional modeling of the object can be performed. A detailed description of the three-dimensional modeling can be easily understood by those skilled in the art, so that a detailed description thereof will be omitted.

Although illustrative embodiments and applications of the present invention have been shown and described, many changes and modifications may be made without departing from the scope of the present invention, and such modifications may be made by one of ordinary skill in the art to which the present invention pertains It can be clearly understood. Accordingly, the described embodiments are illustrative and not restrictive, and the invention is not limited by the accompanying detailed description, but is capable of modifications within the scope of the claims.

100: three-dimensional scanner 110: housing part
120: chamber section 130: a plurality of image sensor sections
140: Imaging processor 150:
151: Input / output interface module 152: Judgment module
153: storage module

Claims (5)

In a three-dimensional scanner using a multi-camera,
A housing part;
A chamber part located inside the housing part and configured to include a base part and a plurality of side parts and to position an object to be scanned in a scanning space including the base part and the plurality of side parts;
A plurality of image sensor units provided on the plurality of side portions and configured to capture the object placed in the chamber portion;
An image processing unit for processing the image information sensed by the plurality of image sensor units to generate a three-dimensional image; And
And a control unit for controlling the plurality of image sensor units and the image processing unit,
Wherein each of the plurality of image sensor units comprises at least two or more cameras,
Wherein the control unit controls the image sensor unit to again pick up image information that has not yet met the predefined condition among the image information captured by the plurality of image sensor units. Dimensional scanner. The method according to claim 1,
Wherein the plurality of image sensor units comprise:
A first sensor module including a first camera and a second camera provided on a first side surface of the plurality of side portions;
A second sensor module including a third camera and a fourth camera provided on a second side surface of the plurality of side portions;
A third sensor module including a fifth camera and a sixth camera installed on a third side surface of the plurality of side portions; And
And a fourth sensor module including a seventh camera and an eighth camera provided on a fourth side surface of the plurality of side portions. 3. The method of claim 2,
Wherein the control unit controls the first sensor module, the second sensor module, the third sensor module, and the fourth sensor module to continuously capture the object continuously at least three times or more, 3D scanners used. The method of claim 3,
The image processing unit selects one piece of image information for each sensor module among the first to fourth plural pieces of image information continuously picked up by the first to fourth sensor modules according to predefined conditions, The information on the sensor module and the selected image information are matched and transmitted to the control unit as the first to fourth sensing information,
Wherein the control unit controls the multi-camera to again pick up the selected sensor module from among the first to fourth sensor modules based on the first to fourth sensing information, 3D scanners used. 5. The method of claim 4,
Wherein the image processing unit compares first image information and second image information among the plurality of image information, compares the second image information with third image information, and compares the third image information and the first image information And the image information is selected based on the pixel difference value.

Images