KR101781994B1 - Apparatus of multi stereo camera for real time 3D - Google Patents

Abstract

The present invention relates to a free-viewpoint real-time 3D implementing device using a multi-stereo camera, and more particularly, to a free-viewpoint real-time 3D implementing device using a multi-stereo camera to calculate a pixel position of collected images using the multi-stereo camera installed at a multi-angle, thereby providing an image in all directions in real time.

Description

[0001] The present invention relates to a stereoscopic 3D camera,

The present invention relates to a free-time real-time 3D imple- mentation apparatus using a multi-stereo camera, and more particularly, to a real-time 3D imple- mentation apparatus using a multi-stereo camera, Time 3D real-time implementation using multiple stereo cameras.

Generally, in order to generate three-dimensional data, there is a method of calculating a relation between images obtained by installing a plurality of cameras and a method of calculating overlapping between images while moving the camera, And a method of calculating the parallax between two images using a stereo camera.

Among these methods, there is a disadvantage in that errors are accumulated as moving images are taken while moving from a reference image by overlapping while moving a camera.

On the other hand, 3D signal processing or information compression technology has been developed and 3D image analysis or contents have been rapidly developed and popularized. The processing and display technology of three-dimensional image is not limited to broadcasting, Has been widely applied.

However, in spite of these technological developments, the general CCTV application can still see only one scene of 3D space even if it is watched from one side to the other.

In addition, most of the existing security markets are single-image-based CCTV, and most of the intelligent CCTV fields analyzing CCTV images are based on single images.

Therefore, 3 - D modeling is needed to solve the disadvantages of 2D analysis, but it still has difficulties in real - time 3D modeling in terms of quality and speed.

In addition, the conventional free view image technology requires a large number of cameras for overlapping images for 3D image generation, and it is difficult to realize 3D images in 360 degrees direction in real time.

Korean Patent Publication No. 10-2008-0068438 (Jul. 23, 2008) 'Method of Generating 3D Terrain Model and 3D Surface Model in Stereo Image'

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned problems in the prior art, and it is an object of the present invention to provide a multi- Time 3D realization device using a multi-stereoscopic camera.

According to an aspect of the present invention, there is provided an image processing apparatus comprising: an image processing controller (310) connected to and communicating with a stereo camera (100) for generating a stereoscopic image information signal and receiving image information through a wireless communication network; An image database 350 connected to the image processing controller 310 and storing the received image information; A stereo matching unit 320 for extracting a stereo image stored in the image database 350 and matching according to a control signal of the image processing controller 310; A point cloud generator 330 for generating a point cloud for acquiring a three-dimensional image as a matched image according to a control signal of the image processing controller 310, and a point cloud generator 330 for generating a point cloud And a 3D modeling unit 340 for generating a three-dimensional image, the apparatus comprising: The stereoscopic camera 100 includes a communication module for wireless communication and is mounted on a fixed unit 200 to photograph a stereo image. The fixed unit 200 includes a base plate 210 having a rectangular plate shape An insertion groove 212 having a predetermined depth is formed in the upper surface of the base plate 210. A stator FFT is fixed to both sides of the short side of the insertion groove 212, A fixed latch FRT is fixed to both sides of the cylinder 220 corresponding to the FFT and a floating latch MRT is rotatably connected to the hinge axis RT about the fixed latch FRT, A torsion spring TSP fitted to the hinge axis RT is provided between the fixed latch FRT and the floating latch MRT so that the floating latch MRT is installed to be elastically rotatable relative to the fixed latch FRT , And one side surface of the elevation guide cylinder 220 A guide block 222 having a predetermined length is formed in a rectangular shape and a flow block 230 is inserted into the lift guide barrel 220. A hook 232 protrudes from the center of the lower end face of the flow block 230 A camera body 270 having a stereo camera 100 is detachably assembled to the upper surface of the flow block 230. A diaphragm 224 is fixed to a lower portion of the elevation guide barrel 220 An elevating motor 226 is installed on the inner bottom surface of the partition 224 and an elevating drum 228 is mounted on the elevating motor 226 to receive the power of the elevating motor 226. And a wire WR is wound around the lifting drum 228. The end of the wire WR passes through the partition 224 and is coupled to the hook 232 of the flow block 230, A lower end of an elastic spring SPR is fixed to an upper surface of the diaphragm 224, And the camera body 270 is lifted and lowered along the guide groove 222. The 3D real-time 3D realization device using the multi-stereoscopic camera do.

According to the present invention, it is possible to obtain an image in a 360-degree direction in real time by calculating pixel positions of images collected using multiple stereo cameras installed at various angles.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is an illustration showing an original stereoscopic image before epipolar geometric correction according to the present invention. FIG.
2 is a diagram illustrating an example of a stereoscopic image after epipolar geometric correction according to the present invention.
FIG. 3 is a diagram illustrating a state in which a depth map is generated in real time using a stereo camera according to the present invention.
4 is an exemplary diagram showing a state in which a three-dimensional point cloud is generated.
5 is an exemplary diagram showing a state in which consecutive point cloud data superimposition and an outlier are removed.
6 is another example of a state in which continuous point cloud data overlap and an abnormal point are removed.
FIG. 7 is an exemplary view showing a state in which a stereo camera according to the present invention is installed.
8 is an exemplary diagram showing a point cloud state generated by each stereo camera.
FIG. 9 is an exemplary block diagram of a free-time real-time 3D imple- mentation apparatus using a multi-stereo camera according to the present invention.
Fig. 10 is an exemplary view of a fixing unit for fixing the stereo camera of Fig. 9;
Fig. 11 is a schematic diagram showing the main part of Fig.
12 is an exemplary view showing the lifting guide barrel locking means of Fig.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Before describing the present invention, the following specific structural or functional descriptions are merely illustrative for the purpose of describing an embodiment according to the concept of the present invention, and embodiments according to the concept of the present invention may be embodied in various forms, And should not be construed as limited to the embodiments described herein.

In addition, since the embodiments according to the concept of the present invention can make various changes and have various forms, specific embodiments are illustrated in the drawings and described in detail herein. However, it should be understood that the embodiments according to the concept of the present invention are not intended to limit the present invention to specific modes of operation, but include all modifications, equivalents and alternatives falling within the spirit and scope of the present invention.

The most significant feature of the present invention is that three-dimensional data is generated in real time using a multi-stereo camera and installed in many directions and viewed at 360 degrees free time, thereby compensating for the shortcomings of the two-dimensional image.

The free-time real-time 3D imple- mentation device using the multi-angle (stereo) camera according to the present invention converts the real-based 3D point cloud data generated in real time in each stereo camera into the modeling space, There is an advantage that matching is not required.

The main point of the present invention is that a stereo camera is installed in consideration of a position and an attitude value by uniformly overlapping a target space to be observed, synthesizes point cloud data generated by each stereo camera, Transforming it into a modeling space, removing anomalous points, and real-time 3D modeling so that it can be viewed as a free time.

The method of generating the three-dimensional data by calculating the relationship between the images by installing the camera in multiple or by repeatedly shooting the image requires continuously generating the three-dimensional data by calculating the relationship between each image, while the stereo camera is fixed After calculating the geometric relations of the two cameras in advance, it is possible to generate three-dimensional data by calculating only the time difference, and real-space-based three-dimensional data can be generated in real time using the parameters of the camera.

Typically, the free-point camera relay technology announced by Canon uses a method of modeling and expressing only a predetermined number of athletes in real time after preliminarily modeling an unchanging background space (soccer field).

In this method, since the 3D modeling of all the spaces can not be performed in real time due to the limitation of the processing speed, the background space is generally changed continuously and there is a restriction in the real life area where the shape and size of the object to be modeled are not fixed.

However, the present invention is a technology for generating a real-based 3D point cloud using a stereo camera, and simultaneously expressing the generated point clouds in a modeling space.

Therefore, these techniques can be actively applied in areas where real-time 360-degree 3D modeling is required, sports arenas where 3D-modeled objects need to be observed at real-time free time, and security vulnerable areas requiring 360-degree monitoring.

For example, although the exemplary screens of FIGS. 1 and 2 are not many, the stereoscopic camera, which knows the position and the posture every time the stereoscopic camera is photographed while moving, is expressed in the same modeling space. Technically, 2 is an image after the epipolar geometric correction, and Fig.

3 shows a state in which a depth map is generated in real time using a stereo camera. A state in which a three-dimensional point cloud is generated through the depth map is shown in FIG. 4 .

After that, the state in which consecutive point cloud data superimposition and outlier are removed is shown in Fig. 5, and another example of a state in which consecutive point cloud data overlap and an abnormal point are removed is shown in Fig.

In other words, the present invention is installed so that a stereo camera can be repeatedly photographed around a desired object space. The exact position and direction of the stereo camera are determined at the time of installation, and the state can be illustrated as shown in FIG.

In the example of FIG. 7, each stereo camera processes the photographed image to generate point cloud data, and the generated point cloud state is the same as the example of FIG. 8, and sets the 3D modeling space with the generated point cloud data.

In addition, the point cloud data from each stereo camera is converted into the coordinates of the modeling space using the installation position and orientation of the stereo camera.

Then, the point cloud data superimposition and the outlier of each direction coordinate transformed are removed.

Here, the epipolar geometric correction is a technique of generating an epipolar curve through a repetitive projection on a left image and a ground-point right image starting from an arbitrary point on the left image in a stereo camera, and geometrically correcting the generated epipolar curve .

In addition, point cloud generation is a result of active research in the field of computer vision to obtain three-dimensional information using two-dimensional images. Most of the basic researches refer to partial three-dimensional restoration through stereo geometry research in a stereo camera .

The most commonly used technique in this field is Structure from Motion (SFM) using multiple image geometry through three or more images. This SfM technique uses three-dimensional point cloud It is a computer vision technique for reconstructing a subject with data. It is a useful technique that has a three-dimensional correction function by processing an image taken without a special condition in advance, .

In this case, the image used in the present invention is a stereo-matched image obtained through a stereo camera.

In this case, the stereo matching is a process of finding one point in the reference image and one point in the reference image in the target image because there are two left eye images and two right eye images due to the characteristics of the stereo camera. In this paper, we propose a new method of matching, which consists of multiple matching methods using multiple matching elements, a matching method using color information, a matching method using multiple stereo images, a matching method using phase, and a method using neural network , A method using a genetic algorithm, etc. are known, so that a reasonable one of them can be adopted and used.

As shown in FIG. 9, an apparatus for real-time real-time 3D rendering using a multi-stereoscopic camera according to the present invention for realizing such a concept comprises an image processing unit 110 for communicating with a stereo camera 100 via a wireless communication network, A controller 310; An image database 350 connected to the image processing controller 310 and storing the received image information; A stereo matching unit 320 for extracting the stereo image stored in the image database 350 and matching according to a control signal of the image processing controller 310; A point cloud generator 330 for generating a point cloud for acquiring a three-dimensional image as a matched image according to a control signal of the image processing controller 310, and a point cloud generator 330 for generating a point cloud And a 3D modeling unit 340 for generating a three-dimensional image.

At this time, the final image modeled as the three-dimensional image is stored in the image database 350 under the control of the image processing controller 310.

The stereo camera 100 includes a communication module to enable wireless communication. The stereo camera 100 includes a fixed unit 200 (see FIG. 10 and FIG. 11) so as to adjust a height and a photographing angle while maintaining stability at an installation site. To be fixedly used.

In this case, it is particularly preferable that the fixing unit 200 is constructed so as to reduce a long service life and maintenance and replacement costs by molding the same with a synthetic resin composition having excellent corrosion resistance, foreign matter pollution prevention property, salt resistance and durability.

The fixed unit 200 includes a base plate 210 in a rectangular plate shape.

The upper surface of the base plate 210 is detachably fixed to the lifting and lowering guide cylinder 220 in a rectangular tube shape.

To this end, an insertion groove 212 having a predetermined depth is formed on the upper surface of the base plate 210, and a stator (FFT) is fixedly protruded on both sides of the short side of the insertion groove 212 .

A fixed latch FRT is fixed to both sides of the lifting and guiding cylinder 220. A floating latch MRT is rotatably connected to the fixed latch FRT about a hinge axis RT, A torsion spring TSP fitted to the hinge axis RT is provided between the fixed latch FRT and the floating latch MRT so that the floating latch MRT is installed to be elastically rotatable with respect to the fixed latch FRT .

When the elevation guide cylinder 220 is fixed to the base plate 210, a part of the lower end of the elevation guide cylinder 220 is inserted into the insertion groove 212 of the base plate 210, The front end of the floating latch MRT is pressed to the FFT so that the stator FFT is elastically deformed and the front end of the instantaneous floating latch MRT is fixed to the stator FFT ), And is hooked.

Therefore, when the movable latch (MRT) is lifted up at the time of detachment, the lower side of the protruding portion of the stator (FFT) is chamfered, so that it is naturally pushed and detached.

A guide groove 222 is formed at one side of the lifting and lowering guide barrel 220 and is cut into a rectangular shape with a predetermined length from the upper end of the lifting guide barrel 220. A flow block 230 is inserted into the lifting guide barrel 220.

A hook 232 protrudes from the center of the bottom surface of the flow block 230 and a fixed hole 234 protrudes from the top of the flow block 230 at a predetermined height.

A pair of support pieces 242 parallel to each other are protruded from the upper end of the fitting bar 240, The fitting piece 252 is fitted in the fitting hole 242, and is then fixed by a hinge pin 260 so as to be rotatable.

At this time, a thread of 2-3 pitch is formed around the hinge pin 260 only at the tip end and the rear end part, and the through hole is formed in the through hole bore formed in the support piece 242 after passing through the support piece 242. And screwed together with the threads of the pitch.

A curved portion UR is formed in the shape of a wavy pattern in the circumferential direction at the center of the length between the threads and the threads of the hinge pin 260. A semicircular locking portion 252 is formed in a part of the inner diameter of the hole formed in the fitting piece 252, (DR) is formed.

In this case, since the semicircular latching portion DR and the bending portion UR are fine, they can be rotated in a multi-step engagement state when they are turned by applying a little force, but they can be held in a locked state when they are not applied with external force .

A fixing rod 250 is integrally formed on the upper end of the fitting piece 252 and the upper end of the fixing rod 250 is fixed to the lower end surface of the camera body 270 having the stereo camera 100 do.

Therefore, the camera body 270 can be rotated in the front-rear direction by the hinge pin 260, so that there is no problem in adjusting the photographing angle.

11, a diaphragm 224 is fixed to a lower side of the cylinder of the elevation guide cylinder 220. The lower portion of the diaphragm 224 is also held in an empty space, An elevating motor 226 is installed in the elevating motor 226 and an elevating drum 228 is mounted on the elevating motor 226 to receive rotation of the elevating motor 226. The elevating drum 228 is wound with a wire WR, (WR) passes through the diaphragm (224) and is then bound to the hook (232) of the flow block (230).

Therefore, when the elevating motor 226 is rotated forward and the elevating drum 228 winds the wire WR, the moving block 230 is lowered.

A lower end of an elastic spring SPR is fixed to the upper surface of the diaphragm 224 and an upper end of the elastic spring SPR is fixed to a lower surface of the flow block 230.

At this time, the wire WR is disposed inside the elastic spring SPR, that is, in the inner hollow space.

Therefore, the flow block 230 always moves upward due to the elastic spring SPR, and the wire WR pulls the flow block 230 to adjust the position (height) of the flow block 230.

Particularly, since the guide groove 222 is formed in one side of the elevation guide barrel 220 in a slot shape, there is no problem that the stereo camera 100 assembled in the flow block 230 moves upward and downward, Even when the angle is adjusted in the vertical direction, the width of the guide groove 222 is larger than the width of the stereo camera 100, which is not a problem at all and is not interfered or interfered with at the time of photographing.

In addition, when a scale RUL is formed on the surface of one elevation guide barrel 220 on which the guide groove 222 is formed, when the stereoscopic camera 100 is used to photograph a plurality of elevation guide cylinders 220, ) To be kept in coincidence with one another.

When the photographing position of the stereo camera 100 is set by using such a configuration, a plurality of base plates 210 are disposed at the photographing position, and then the elevating motor 228 is rotated forward or reverse to move the position of the moving block 230 .

Then, if the camera body 270 is held at a predetermined angle around the hinge pin 260 to adjust the photographing angle of the stereo camera 100, a plurality of stereo cameras 100 are simultaneously It is based on making 3D images.

In addition, the base plate 210, the elevation guide cylinder 220, and the flow block 230 are used by molding the following synthetic resin composition.

In this case, the synthetic resin composition comprises 60 wt% of polyethylene resin, 20 wt% of silica, and the remaining functional additive, wherein the functional additive comprises, based on 100 parts by weight of the polyethylene resin, 2 parts by weight of 1,4- 3 parts by weight of titanate, 1.5 parts by weight of Mica, 3 parts by weight of colloidal silica, 4 parts by weight of aluminum alkoxide, 10 parts by weight of polylactic acid, 2 parts by weight of alumina powder having a particle size of 1-2 μm, 2 parts of N-cyclohexybenzothiazole-2-sulfenamide, 3 parts of montmorillonite, 7 parts of Ds (dichlorodimethylsilane), 3 parts of calcium stearate, 4 parts of ethylene glycol monomethyl ether, 5 parts of Polysorbate 80 5 parts by weight of sodium silicate, 5 parts by weight of sodium erosorbate, 3 parts by weight of salicylic acid ester, 2 parts by weight of terbium, 5 parts by weight of bauxite clinker powder having a particle size of 0.1-0.2 μm, 2 parts by weight of aminopropyltriethoxysilane, 2 parts by weight of aromatic polyamine, 1 part by weight of sodium oxide (Na ₂ O), 1 part by weight of ferric trioxide (Fe ₂ O ₃ ), 2 parts by weight of alkylene amide, And 2 parts by weight of a water lily powder having a particle size of 0.1 mm or less.

At this time, the polyethylene resin is a base resin, and the 1,4-butylene glycol is added in order to enhance adhesion and maximize adhesion, and the tetraisopropyltitanate is a coupling agent having an organic titanate structure, It is added to enhance the durability by strengthening the interfacial adhesion force between the resin and the inorganic material.

The mica is a kind of silicate minerals, which is crushed to a size of 0.1-0.2 μm and sieved, and is added to enhance durability and impact resistance due to the hardness characteristics.

In addition, the colloidal silica is preferably an amorphous silica sol having a particle diameter of 5-50 nm, and the aluminum alkoxide is added to perform a curing catalyst and a crosslinking agent function, and a secondary coating having a high hardness Contributing.

In addition, the polylactic acid is a synthetic polymer type resin having excellent moisture resistance and processability, has a melting temperature of 150 to 200 ° C, and is improved in softness, tensile strength and elongation by ductility.

And, alumina is a compound of aluminum and oxygen and added for improving durability.

In addition, CZ (N-cyclohexybenzothiazole-2-sulfenamide) is added to increase surface slip property and to maximize prevention of foreign matter adhesion.

In addition, montmorillonite is added as an inorganic filler to increase mechanical properties, and Ds (Dichlorodimethylsilane) is added as a strong hydrophobic substance to maximize moisture resistance.

And, calcium stearate is added as a dispersant to promote the lubricating function and induce homogeneous dispersibility of the additives.

In addition, ethylene glycol monomethyl ether is added to enhance the adhesiveness, and it is added to increase the durability by strengthening the adhesion force.

In addition, Polysorbate 80 (Polysorbate 80) is one of the nonionic surfactants derived from sorbitol, which is added to prevent moisture from spreading. Sodium silicates are added to enhance the surface adhesion and enhance the cohesiveness. And sodium erosorbate is added to prevent oxidation.

In addition, the salicylic acid ester has a function of absorbing ultraviolet rays to prevent the resin from being deformed by ultraviolet rays.

In addition, terbium is a rare earth metal belonging to lanthanum family, and it has a good ductility and ductility, which contributes to improvement of buffering and wear resistance of the coating layer.

In addition, a bauxite clinker powder having a particle size of 0.1-0.2 mu m is added to increase the binding force to increase the compressive strength.

The? -Aminopropyltriethoxysilane is added as a silane coupling agent for coupling between a thermosetting resin such as a melamine resin and an inorganic material in order to increase the bonding property, the adhesion property, and the surface strength.

In addition, the aromatic polyamines are intended to promote curing.

In addition, the sodium oxide performs a function of raising the refractivity by reacting with silicon dioxide to form a silicate. In particular, the sodium oxide also performs an antioxidant function.

The iron diiron trioxide is mainly used for anti-corrosive function, but it is added in order to suppress interfacial separation in the present invention, and it should be added in a small amount due to the characteristic of iron oxide.

In addition, the alkylene amide is added to maintain lubricity and stability, and is added in order to smoothly mix and prevent crushing after mixing.

In addition, the sulfuric acid ester is added in order to increase the heat resistance while being decomposed by the photo-thermal effect for ultraviolet rays.

Furthermore, the water lily powder is obtained by taking a water lily, drying it thoroughly, and pulverizing it into a powder, which is a natural moisture absorber and biodegradable, thus being eco-friendly. The training has an excellent effect especially for adsorbing hydrocarbons and oil mist.

In order to confirm the foreign matter resistance and the corrosion resistance (moisture resistance) according to such composition, it is molded into a sheet of 10 cm × 5 cm × 0.5 cm size, and then water is sprayed on the surface, Respectively. As a result of the experiment, it was confirmed that the water flowed down after 15 degrees and the humidity was excellent. This also means that the foreign matter prevention property is high.

100: Stereo camera 210: Base plate
220: lift-up guide bar 230:

Claims (1)

An image processing controller (310) connected to and communicating with a stereo camera (100) for generating a stereoscopic image information signal through a wireless communication network and receiving image information; An image database 350 connected to the image processing controller 310 and storing the received image information; A stereo matching unit 320 for extracting a stereo image stored in the image database 350 and matching according to a control signal of the image processing controller 310; A point cloud generator 330 for generating a point cloud for acquiring a three-dimensional image as a matched image according to a control signal of the image processing controller 310, and a point cloud generator 330 for generating a point cloud And a 3D modeling unit 340 for generating a three-dimensional image, the apparatus comprising:
The stereo camera 100 includes a communication module for enabling wireless communication, and is configured to be fixedly mounted on the fixed unit 200 to photograph a stereo image,
The fixed unit 200 includes a base plate 210 having a rectangular plate shape and an insertion groove 212 having a predetermined depth is formed on the upper surface of the base plate 210, And a fixed latch FRT is fixed to both sides of the lifting and guiding cylinder 220 corresponding to the FFT of the lifting and guiding cylinder 220. The fixed latch FRT is provided with a floating latch MRT, And a torsion spring TSP fitted to the hinge axis RT is provided between the fixed latch FRT and the floating latch MRT so as to be rotatable about the axis RT, A guide groove 222 is formed at one side of the elevation guide barrel 220 and is cut into a square shape with a predetermined length from the upper end of the elevation guide barrel 220. In the elevation guide barrel 220, The flow block 230 is inserted into the center of the bottom surface of the flow block 230, A camera body 270 having a stereo camera 100 is detachably assembled to the upper surface of the flow block 230 and a lower portion of the diaphragm A lifting motor 226 is installed on the inner bottom surface of the partition plate 224 and a lifting motor 226 is mounted on the lifting motor 226 to lift and lower And a wire WR is wound around the lifting drum 228. The end of the wire WR passes through the partition plate 224 and then passes through the hooks 232 of the flow block 230. [ A lower end of an elastic spring SPR is fixed to an upper surface of the partition plate 224 and an upper end of the elastic spring SPR is fixed to a lower surface of the flow block 230, Is moved up and down along the guide groove (222) Freeview device using real-time 3D implementation.

Images