KR101429607B1 - An Adaptive Switching Method for Three-Dimensional Hybrid Cameras - Google Patents

Abstract

The present invention relates to a method to adaptively switch a hybrid stereoscopic camera, whereby a hybrid camera is configured by combining a manual camera and an active camera, which adaptively switches between the manual and active cameras of the hybrid camera in accordance to the complexity of the surroundings. The method comprises the steps of: (a) configuring a hybrid camera by including an active camera and a manual camera; (b) receiving an initial image; (c) converting the initial image into an HIS color space and extracting brightness characteristics from a brightness image; (d) extracting texture characteristics using Gabor wavelet transform for the initial image; (e) creating an integrated criterion using a weighted sum of the brightness characteristics and the texture characteristics; and (f) comparing the integrated criterion with a predetermined threshold value and switching the mode of a hybrid camera in accordance to the comparison results. The method to adaptively switch a hybrid stereoscopic camera described above can adaptively switch between the manual and active cameras of the hybrid camera in accordance to the levels of the brightness characteristics and the texture characteristics indicating the complexity of the surroundings, thereby extracting three-dimensional image information more robustly.

Description

An Adaptive Switching Method for Three-Dimensional Hybrid Cameras

The present invention relates to an adaptive switching method of a hybrid stereoscopic camera capable of more accurately extracting three-dimensional image information by adaptively switching a passive type and an active type camera of a hybrid camera according to the complexity of the surrounding environment.

Generally, the 3D stereoscopic display uses depth-of-view stereoscopic vision technology to add depth information to a 2D image. By using this depth information, viewers can feel the feeling of being in a scene where a video is being produced It is the next generation new technology that makes you feel.

This technology is very diverse in applications such as information and communication, broadcasting, medical, education and training, military, game, animation, hologram, virtual reality, CAD, and industrial technology [Non-Patent Documents 1 and 2]. Currently, the market interest in the 3D industry is explosive. There are lots of issues that need to be solved such as standardization and securing the stability of technology. However, there is no question that 3D is the mainstream industry in spite of these problems.

One of the key element technologies for effectively implementing such a three-dimensional solid object is a three-dimensional camera processing technology. Generally, a three-dimensional stereoscopic camera is largely classified into a time-of-flight (non-patent document 3), a passive triangulation (non-patent document 4), and an active triangulation Patent Document 5].

The TOF method is a method of measuring the distance by the difference of time of returning the energy such as the ultrasonic wave or the light wave such as the laser to the object and it is difficult to interpret the sound wave even if the environment is changed a little, have.

The passive trigonometric method is one of the most widely used methods, but there is a disadvantage that distance information can not be obtained because a corresponding point can not be determined in an object such as a wall surface where no texture exists.

And the active trigonometry can be interpreted geometrically as coping with a projector with a lens on a stereo camera, which has the advantage of being able to determine the point of correspondence even in the less textured areas, while the slower It has disadvantages.

In this way, most existing 3D cameras shoot scenes using one type of camera fixedly, so there is a limit to the image quality. Therefore, there is a need for a technique that can extract three-dimensional image information more accurately by using a camera according to the complexity of the surrounding environment by combining these cameras.

[Non-Patent Document 1] A. Smolic, "3D Video and Free Viewpoint Video- From Capture to Display," Pattern Recognition, Vol. 44, No. 9, pp. 1958-1968, Sep. 2011. [Non-Patent Document 2] H.-J. Choi, Y.-H. Seo, S.-W. Jang, and D.-W. Kim, "Analysis of Digital Hologram Rendering Using a Computational Method," Journal of Information and Communication Convergence Engineering, Vol. 10, No. 2, pp. 205-209, Jun. 2012. [Non-Patent Document 3] Y. Cui, S. Schuon, D. Chan, S. Thrun, and C. Theobalt, "3D Shape Scanning with a Time-Of-Flight Camera," IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1173-1180, 2010. [Non-Patent Document 4] F. Dornaika, A. D. Sappa, "Featureless and Stochastic Approach to On-Board Stereo Vision System Pose," Image and Vision Computing, Vol. 27, No. 9, pp. 1382-1393, Aug. 2009. [Non-Patent Document 5] S.-Y. Lee and S.-W. Jang, "A Study on Image Processing for Accuracy Improvement of 3D Recovery," In Proc. of the Society of Computer and Information, Vol. 20, No. 1, pp. 193-195, Jan. 2012. [Non-Patent Document 6] T. Ringbeck and B. Hagebeuker, "A 3D Time-of-light Camera for Object Detection," In Proc. of Optical 3D Measurement Techniques, Plenary Session, July 2007. [Non-Patent Document 7] Mesa Imaging, http://www.mesa-imaging.ch [Non-Patent Document 8] Breuckmann, http://www.breuckmann.com [Non-Patent Document 9] K. Bunte, M. Biehl, M. F. Jonkman, and N. Petkov, "Learning Effective Color Features for Retrieval in Dermatology," Pattern Recognition, Vol. 44, No. 9, pp. 1892- 1902, 2011. [Non-Patent Document 10] S.-W. Jang, M.-A. Chung, G.-Y. Kim, "Face Model-based Image Registration for Generating Facial Textures," In Proc. of the ACM Research in Applied Computation Symposium, pp. 121- 125, October 2012.

SUMMARY OF THE INVENTION The object of the present invention is to solve the above problems and to provide a hybrid camera in which a passive camera and an active camera are combined to constitute a hybrid camera and adaptively adapt passive and active cameras of a hybrid camera And a switching device for switching the three-dimensional image information from the three-dimensional image information.

It is also an object of the present invention to first acquire an initial image, to extract a brightness feature and a texture feature, which are main features representative of the surrounding environment, from the input image, The present invention provides an adaptive switching method of a hybrid stereoscopic camera that adaptively switches a hybrid camera by generating a rule for selecting a camera capable of selecting a camera.

In order to achieve the above object, the present invention is directed to an adaptive switching method of a hybrid stereoscopic camera.

According to another aspect of the present invention, there is provided an adaptive switching method of a hybrid stereoscopic camera, including: (a) configuring a hybrid camera including an active camera and a passive camera; (b) receiving an initial image; (c) converting the initial image into an HIS color space, and then extracting brightness characteristics of the brightness image; (d) extracting a texture feature using the wavelet transform on the initial image; (e) generating an integrated measure using the weighted sum of the brightness feature and the texture feature; And (f) switching the hybrid camera according to the contrast result, comparing the integrated measure with a predetermined threshold value.

According to another aspect of the present invention, there is provided an adaptive switching method for a hybrid stereoscopic camera, comprising the steps of: dividing the initial image into blocks each having a size of N by N pixels in step (c) or step (d) .

According to another aspect of the present invention, there is provided an adaptive switching method for a hybrid stereoscopic camera, wherein the brightness characteristic is normalized to have a value between 0 and 1 in step (c).

According to another aspect of the present invention, in the adaptive switching method of a hybrid stereoscopic camera, in the step (d), the Gabor wavelet coefficient is calculated by multiplying brightness values in one block by a wavelet function.

According to another aspect of the present invention, there is provided an adaptive switching method of a hybrid stereoscopic camera, wherein in the step (d), all coordinates of an area represented by each block are calculated by using a window centered on the coordinate, And the average is extracted as the texture characteristic of each block.

In the adaptive switching method of a hybrid stereoscopic camera according to the present invention, in the step (e), the integrated measure Φ (t; α, β) for the initial image at time t is defined by the following equation .

[Equation 1]

Where F _bright (t) and F _gabor (t) are the brightness and texture characteristics, respectively.

According to another aspect of the present invention, there is provided an adaptive switching method of a hybrid stereoscopic camera, wherein, in the step (f), the passive camera is activated when the integrated scale is greater than or equal to a predetermined threshold value, .

In the adaptive switching method of a hybrid stereoscopic camera according to the present invention, in the step (e), the integrated measure Φ (t; α, β) for the initial image at time t is defined by the following equation .

[Equation 2]

Where F _bright (t) and F _gabor (t) are the brightness and texture characteristics, respectively.

According to another aspect of the present invention, there is provided an adaptive switching method of a hybrid stereoscopic camera, wherein, in the step (f), the active camera is activated when the integrated scale is equal to or greater than a predetermined threshold, .

In addition, the present invention relates to a computer-readable recording medium on which a program for performing an adaptive switching method of a hybrid stereoscopic camera is recorded.

As described above, according to the adaptive switching method of the hybrid stereoscopic camera according to the present invention, the adaptive switching of the passive type and the active type camera of the hybrid camera according to the brightness characteristic and the degree of the texture characteristic, It is possible to obtain a more robust image information.

As a result of carrying out the experiment by implementing the method according to the present invention, it was found that the present invention operates reliably under various conditions and can be practically applied to a hybrid camera capable of replacing a conventional three-dimensional camera using only an active or passive camera As shown in FIG.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram showing a configuration of an overall system for carrying out the present invention; Fig.
2 is a flowchart illustrating an adaptive switching method of a hybrid stereoscopic camera according to the first embodiment of the present invention.
Fig. 3 is an exemplary view of a TOF camera, (a) an example of a camera of PMD CamCube (b) SwissRanger 4000;
Fig. 4 is a perspective view of a passive camera used in the present invention. Fig. 4 (a) is a Bumblebee camera, and Fig. 4 (b) is an example of a Videre camera.
Fig. 5 is an exemplary view of a smartSCAN3D-HE, which is an active camera used in the present invention. Fig.
6 is a diagram illustrating an HIS color space according to the present invention.
7 is a graph showing the items of the brightness feature and the texture feature according to the first embodiment of the present invention.
8 is a pseudo code illustrating the steps of switching the passive camera and the active camera according to the first embodiment of the present invention.
9 is a graph showing items of brightness and texture features according to a second embodiment of the present invention.
10 is a pseudo code illustrating the steps of switching a passive camera and an active camera according to a second embodiment of the present invention;
11 is a table illustrating a camera selected according to the environment in accordance with the present invention;
12 is an image of an exemplary image according to different lighting conditions according to an experiment of the present invention;
13 is an image of an example image according to different texture conditions according to the experiment of the present invention.
14 is an illustration of an example of a passive camera (stereo camera) according to the experiment of the present invention.
15 is an exemplary view of an active camera according to an experiment of the present invention.
16 is a displacement map of an active camera according to an experiment of the present invention, wherein (a) is an image of an input image, and (b) a displacement map.
17 is a displacement map of a passive camera according to the experiment of the present invention, which is a displacement map of (a) an input image (left), (b) a displacement map, (c) Maps.
18 is a table showing the adaptive selection of a hybrid camera according to an experiment of the present invention.
FIG. 19 is a graph comparing average execution time ratios of the conventional method and the method according to the present invention, according to the experiment of the present invention. FIG.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the drawings.

In the description of the present invention, the same parts are denoted by the same reference numerals, and repetitive description thereof will be omitted.

First, examples of the configuration of the entire system for carrying out the present invention will be described with reference to Fig.

1, an adaptive switching method of a hybrid stereoscopic camera according to the present invention is a method of selecting a passive camera 21 and an active camera 22 to receive a stereoscopic image of an object 10 from a selected camera May be implemented as a program system on the computer terminal 30. That is, the adaptive switching method can be implemented by a program and installed in the computer terminal 30 and executed. The program installed in the computer terminal 30 can operate as one program system 40. [

Meanwhile, as another embodiment, the adaptive switching method of the hybrid stereoscopic camera may be implemented by a single electronic circuit such as an ASIC (application-specific semiconductor) in addition to being operated by a general-purpose computer. Or a dedicated computer terminal 20 dedicated to only selecting one of the passive camera 21 and the active camera 22. [ This is referred to as an adaptive switching device 40. Other possible forms may also be practiced.

The passive camera 21 is a camera that acquires distance information from a computer vision using a stereo camera, that is, the parallax of two cameras, such as a human eye. The photographed image 61 is an image photographed by the passive type camera 21 and is referred to as a passive type image 61.

The active camera 21 refers to a device for projecting a coded pattern image successively using a projector and a light source on one side and acquiring an image of a scene on which the pattern is projected on the other side. The image obtained from the active type camera 21 will be referred to as an active type image 62.

The passive type image or the active type image 61,62 is composed of consecutive frames in time. One frame has one image. That is, the image at a specific time t is one frame immediately.

Next, an adaptive switching method of the hybrid stereoscopic camera according to the first embodiment of the present invention will be described with reference to FIG. FIG. 2 shows a general flowchart of an adaptive switching method of a hybrid stereoscopic camera according to the present invention.

As shown in FIG. 2, in the method according to the present invention, a hybrid camera is first constructed (S10), an initial image is received (S20), and a brightness characteristic and a texture The feature is extracted (S30, S40). Then, a scale for selecting a camera that best accommodates the extracted brightness and texture feature values (hereinafter, a switching integration scale) is created (S50), and the hybrid stereoscopic camera is adaptively (S60). Through this, images with better quality are obtained.

Hereinafter, each step will be described in more detail.

First, the configuration step S10 of the hybrid camera will be described in more detail.

Hybrid cameras consist of a passive camera and an active camera. Hereinafter, a three-dimensional stereoscopic camera used in the present invention will be described.

In recent years, various kinds of three-dimensional stereoscopic cameras have been widely used in applications requiring high-quality images. The TOF-based three-dimensional camera [Non-Patent Document 3] is a camera that recognizes the distance to an object by radiating energy such as light wave or sound wave to an object and returning time. Fig. 3 shows a TOF camera ] [Non-Patent Document 7].

The projected energy of the TOF camera uses ultrasonic waves as sound waves and laser light as light waves. In the case of a sensor using an ultrasonic wave, it is difficult to measure at a precise point of a real world because of the propagation characteristics of an ultrasonic wave, and only a typical distance value in a certain angular range of the real world is calculated. Therefore, there is a limit in performing a minute operation such as a hand motion of the robot. In the case of the time delay method using a laser, since it is a one-dimensional sensor type that acquires only distance information at a point of the real world by one measurement, it requires a lot of measurement in order to expand to two or three dimensions. There is a drawback to long.

In this way, the time-delay-based distance sensor can be used to some extent for a large operation such as the movement of the robot, but it is difficult to perform a minute operation such as a robot manipulating the object due to the above-described constraints.

Therefore, a triangulation-based distance measurement method using an image sensor capable of acquiring distance information at a precise point in the real world and simultaneously acquiring distance information at many points within a short time is widely used. This method of acquiring distance information based on trigonometry can be divided into passive type and active type camera depending on whether energy is projected or not.

A passive camera [Non-Patent Document 4] corresponds to a stereo camera in a computer vision, and in stereovision, distance information is acquired using the parallax of two cameras as in a human eye. In Stereo Vision, there is a serious disadvantage that it can not determine the correspondence point in an object such as a wall surface on which no texture exists and thus can not obtain distance information. In the method of acquiring distance information based on trigonometry, distance information is acquired by using the same point relationship of the real world at two points of view. Therefore, it is necessary to solve the point of correspondence to the same points in the real world. In Stereo Vision, there is also a disadvantage that many operations are required to compare and analyze the correlation between two images in order to solve the problem of corresponding points. FIG. 4 shows a Bumblebee camera and a Videre camera which are widely used as a passive camera.

An active camera, that is, a camera based on an active trigonometry [Non-Patent Document 5], is a system in which one of two cameras of a stereo vision system is replaced with a device (laser, projector) capable of projecting light. In other words, the active camera is a method of easily solving the problem of the correspondence point by providing a correspondence point in stereovision by projecting light of known geometric pattern called structured light. The distance measurement method based on structured light is advantageous in that it can calculate a large amount of distance data by providing a small amount of computation and a large number of correspondence points in order to solve the problem of the correspondence point as compared with the stereoscopic vision. There is a disadvantage that it takes more time. Figure 5 shows a smartSCAN3D-HE camera from Breuckmann, Germany, one of the active cameras [Non-Patent Document 8].

Next, the initial image input step S20 will be described.

The initial image is input through the camera (S20). As an initial image, a general color image (or RGB image) is input. Therefore, any one of the stereo cameras of the passive camera can be used as the initial image. Alternatively, as another example, a separate RGB camera may be used to obtain an initial image.

Next, steps S30 and S40 for extracting brightness and texture features will be described. That is, a method for extracting two features representing the complexity of the surrounding environment used in the present invention will be described.

In the present invention, the brightness and the texture characteristic are defined and used as a characteristic representing the state of the surrounding environment. In order to reduce the influence of the noise, the input image is divided into N × N pixel size blocks, and then these features are extracted on a block-by-block basis.

First, the brightness feature extraction step (S30) will be described.

In the brightness feature, the RGB color space of the input image is converted into a HIS color space (Non-Patent Document 9) that can be easily converted into hue, intensity, and saturation, . Equation 1 below expresses an equation for converting an RGB color space into an HIS color space.

[Equation 1]

Here, R, G, and B represent RGB values of the color image, and H, I, and S represent hues, intensities, and saturations, respectively.

 And Figure 6 shows the HIS color space geometrically.

On the other hand, a representation of a color image with only the brightness (I) value is called a brightness image.

Next, the brightness image (or I image) is divided into N × N pixel size blocks, and brightness characteristics are extracted in block units.

In the present invention, the brightness characteristic F _bright (t) for an image at time t is defined as Equation 2.

&Quot; (2) "

Here, N denotes the length of the block in the horizontal or vertical direction, P denotes the total number of blocks sampled in the image, and R _i denotes a set of positional coordinates in the ith block. And I _t (x, y) represents the brightness value at the position of the image (x, y).

In equation 2, the brightness characteristic F _bright (t) of the t-view image is normalized to have a value between 0 and 1.

Next, the step (S40) of extracting the texture feature will be described.

In general, texture features are being used as a key element that characterizes the distribution of pixel values in an image. The texture analysis algorithms used to extract texture features have been studied from random field models to wavelet transform techniques. The feature using Gabor wavelet transform is different from other texture features Has been known to have excellent texture discrimination power and is widely used [Non-Patent Document 10]. Therefore, the present invention also uses a Gabor wavelet transform as a texture feature.

The Gabor wavelet transform is a wavelet transform used as a mother function of a Gabor function. The Gabor function is a Gaussian function modulated by a complex sinusoid function. The two-dimensional Gabor function g (x, y) is defined as Equation 3.

&Quot; (3) "

In Equation 3, σ _x and σ _y are the standard deviations along the x and y axes, respectively, and W is the fundamental frequency of interest.

The Gabor wavelet transform can be expressed as Equation 4 through magnification and rotation of the Gabor function g (x, y).

&Quot; (4) "

In Equation 4, &thetas; = n [pi] / K and K represents the number of possible rotation angles. That is, θ has a value obtained by dividing K by 180 degrees. Therefore, n has an integer value from 0 to K-1. When S is a possible reduction number, m is an integer from 0 to S-1 as a reduction factor.

The Gabor wavelet coefficients are calculated by multiplying the brightness values in one block by a wavelet function.

&Quot; (5) "

W in Equation 5 represents the radius of the window centered at (x, y).

In the present invention, an average of the wavelet coefficients calculated using a window centering on all the coordinates (x, y) of an area represented by the i-th block is used as a texture characteristic of each block.

Next, a description will be given of a step (S50) of generating a switching integrated measure.

After defining the brightness and texture features representing the complexity of the surrounding environment in the previous steps S30 and S40, an integrated measure that effectively combines these features is defined, and adaptive switching based on the defined integrated measure Create a rule.

 Two embodiments for generating a switching integration measure will now be described. First, the switching integration measure according to the first embodiment will be described.

In the present invention, the integrated scale Φ (t; α, β) for effectively measuring the complexity of the surrounding environment is defined as Equation (6) using the weighted sum of the brightness feature and the texture feature.

&Quot; (6) "

In Equation 6, the integrated measure Φ (t; α, β) is defined using a weighted sum of brightness and texture features.

As shown in Figure 7a, if term of the brightness characteristic F _bright (t) is the case have a value between 0 and 1, Φ; and this term is the minimum value of the brightness characteristics of the (t α, β), F bright (t) 0.5 Is defined as the maximum value of the brightness characteristic of? (T;?,?).

And if term of the texture characteristic F _gabor (t) as shown in Figure 7b has a value of 0, Φ; and this term is the minimum value of the texture features of the (t α, β), of the F _gabor (t) 1 (T, α, β) is defined as the maximum value for the texture feature.

In Equation (6), α and β denote the contribution of the brightness feature and the texture feature as a weight. In the present invention, by setting α and β to 0.4 and 0.6, respectively, a higher weight is given to the texture feature than to the brightness feature .

Next, the step S60 of switching the hybrid camera by the generated switching integration measure will be described.

After the integrated metric for measuring the complexity of the surrounding environment is defined, the hybrid stereoscopic camera is dynamically switched by the switching rule (or switching algorithm) shown in FIG. 8 using the generated switching rule (S60).

That is, if the integrated measure? (T;?,?) Is equal to or greater than the threshold value TH, the passive camera of the hybrid camera is activated. Otherwise, the active camera is activated. In the switching rule as shown in Fig. 8, the threshold TH is determined empirically by a repeated experiment as a threshold for switching the camera.

In summary, as shown in FIG. 11, the active camera is activated when the brightness of the surrounding environment is dark or bright, and the texture is weak, and when the brightness and texture of the surrounding environment are not satisfactory, do.

Next, an adaptive switching method of the hybrid stereoscopic camera according to the second embodiment of the present invention will be described with reference to FIGS. 9 and 10. FIG.

The second embodiment of the present invention is basically the same as the first embodiment described above. However, there are some differences in the way to obtain the switching integration measure and the switching rule. Hereinafter, only a part different from the first embodiment will be described.

First, a step S50 of generating a switching integrated measure will be described.

Next, a switching integration measure according to the second embodiment will be described.

The integrated measure? (T;?,?) At the time point t according to the second embodiment is defined as Equation (7).

&Quot; (7) "

Equation 7 is defined by using the weighted sum for the reciprocals, with the inverse of the terms indicating the brightness scale and the terms representing the texture characteristic, as compared with Equation 6 in the first embodiment.

As shown in Figure 9a, if term of the brightness characteristic F _bright (t) is the case have a value of 0.5, Φ; and this term is the minimum value of the brightness characteristics of the (t α, β), F bright (t) are 0 and 1, , The term for the brightness characteristic of? (T;?,?) Is defined to be infinitely larger as the value approaches?

And texture case have a value of the F _gabor (t) 1 as shown in Figure 9b, if term of the characteristics, Φ; to be a term minimum values for the texture features of the (t α, β), F gabor (t) is 0, The closer to the texture property of Φ (t; α, β) is defined to be infinitely large.

Next, the switching step S60 according to the second embodiment will be described.

The switching rule (or switching algorithm) according to the second embodiment is as shown in FIG. That is, the hybrid stereoscopic camera is dynamically switched by the switching rule shown in FIG. 10 (S60). That is, if the integrated measure? (T;?,?) Is equal to or greater than the threshold value TH2, the active camera of the hybrid camera is activated. Otherwise, the passive camera is activated.

The switching rule according to the second embodiment is opposite to that of the first embodiment in that the activated camera is larger than the threshold value and smaller than the threshold value.

By defining the integration scale according to the second embodiment as described above, it is possible to increase the value of the integration scale in the extreme case such as when the texture characteristic is too weak, the brightness is too dark or bright, have.

Specifically, in Fig. 9A, if F _bright (t) is smaller than a or larger than b, the measure term for brightness always becomes larger than the predetermined threshold TH2. Therefore, even if F _gabor (t) increases, the total integrated measure naturally becomes larger than the threshold value TH2. That is, if the brightness is too dark or bright, no matter how sharp the texture is, the total integrated measure always becomes larger than the predetermined threshold, and the active camera is activated.

Conversely, if F _gabor (t) is less than c, then the aggregate measure of F _bright (t), no matter how small (or has a very large value), will naturally fall below the threshold TH2 . That is, in the case of a weak texture, even if the brightness is moderate, the total integrated scale always becomes larger than the predetermined threshold, and the active camera is activated.

On the other hand, in the first embodiment, if the brightness is medium, the passive camera can be activated even when the texture is poor, or conversely, if the texture is clear, the passive camera can be activated even when the brightness is too dark or too bright . The second embodiment can improve such a case.

Next, the effect of the present invention through experiments will be described in more detail with reference to Figs. 12 to 20. Fig.

The computer for the experiment of the present invention uses 2.66GHz CPU and 8GB memory of Intel Pentium Core 2 Duo, and Microsoft Windows 7 was used as the operating system. In order to implement the adaptive switching method of the hybrid stereoscopic camera proposed in the present invention, the integrated development environment of Visual C ++ 2008 was used.

In the experiment of the present invention, the adaptive switching algorithm of the proposed camera is applied in an environment where illumination and texture influence are different. 12 (a), (b), (c), (d), (e), and (f) show the conditions of the illumination as' very bright ',' bright ',' Dark ', and' completely dark '. Similar to FIG. 12, FIGS. 13A, 13B, 13C, 13D, 13E and 13F show texture conditions as' very rich ',' rich ',' 'Poor', 'very poor', and 'poorly'.

In the present invention, a three-dimensional stereo camera, which is a passive type camera, a Bumblebee 2 camera manufactured by Pointgrey, and two single cameras, such as the one shown in FIG. 14, are arranged in parallel to the IEEE 1394 terminal in order to improve the acquired image size and transmission speed. .

As the active camera used in the present invention, Kinect of Microsoft was used as shown in FIG.

FIGS. 16 and 17 show an example of a disparity map extracted by applying an adaptive switching method according to the present invention after receiving an input image in which lighting and texture exist in various ways.

FIG. 16 is an experiment on an image taken in an environment in which the illumination is 'dark' and the texture is 'poor'. The adaptive switching method according to the present invention selects an active camera instead of a passive camera, . And FIG. 17 shows an example of an incorrectly extracted displacement map, which is an experiment on an image taken in an environment where the illumination is 'dark' and the texture is 'very poor'. In FIG. 17, when the passive camera is selected instead of the active camera in an environment in which illumination and texture characteristics are lacking, it is impossible to visually confirm that the displacement map having low accuracy can be acquired because the depth information of three dimensions can not be correctly measured.

FIG. 18 shows the entirety of the three-dimensional camera dynamically selected by the adaptive switching method according to the present invention according to the degree of illumination and texture existing in the surrounding environment. 18, A denotes an active camera, and P denotes a passive camera.

In the experiment of the present invention, in order to compare and evaluate the performance of the adaptive switching method according to the present invention in addition to the visual verification of the 3D displacement map, The information extraction execution time was measured. The execution time is based on 640 × 480 resolution of the image, and the average time ratio of each method is measured as compared with the passive method with the fastest speed as shown in Equation 8.

&Quot; (8) "

FIG. 19 shows a graphical comparison of the average execution time ratio of the conventional method and the method according to the present invention.

As can be seen from FIG. 19, the method according to the present invention has a higher performance than the active type camera, and the method according to the present invention is slightly lower than the passive type Performance. In terms of accuracy, the method according to the present invention is superior to the method using a passive camera and visually confirmed to have a somewhat lower accuracy than the method using an active camera. As a result, it has been determined that the adaptive switching method according to the present invention has good performance to replace the conventional passive or active method in terms of accuracy and time.

In the present invention, a passive camera and an active camera are combined to constitute a hybrid stereoscopic camera. We present a mechanism for robust extraction of 3D image information by adaptively switching the passive and active cameras of hybrid cameras according to the brightness characteristics and the degree of texture features that indicate the complexity of the surrounding environment. In addition, in the experiment, the adaptive switching method proposed in the present invention operates reliably under various conditions and can be practically applied to a hybrid camera that can replace an existing three-dimensional camera using only an active or passive camera alone Respectively.

In the method according to the present invention, the weights are fixedly used in the integrated scale of brightness and texture, but it is not always good for all environments. Therefore, it will be necessary to set the weights more adaptively.

Although the present invention has been described in detail with reference to the above embodiments, it is needless to say that the present invention is not limited to the above-described embodiments, and various modifications may be made without departing from the spirit of the present invention.

10: object
21: Passive camera 22: Active camera
30: computer terminal 40: program system
61: passive image 62: active image

Claims (10)

In an adaptive switching method of a hybrid stereoscopic camera,
(a) constructing a hybrid camera including an active camera and a passive camera;
(b) receiving an initial image;
(c) converting the initial image into an HIS color space, and then extracting brightness characteristics of the brightness image;
(d) extracting a texture feature using the wavelet transform on the initial image;
(e) generating an integrated measure using the weighted sum of the brightness feature and the texture feature; And
(f) activating any one of the active camera and the passive camera according to a contrast result, comparing the integrated scale with a predetermined threshold value, and activating the selected one of the active camera and the passive camera.
The method according to claim 1,
Wherein the step (c) or the step (d) comprises: dividing the initial image into NxN pixel size blocks, and extracting corresponding features in block units.
The method according to claim 1,
Wherein the step (c) normalizes the brightness characteristic to have a value between 0 and 1.
3. The method of claim 2,
Wherein in the step (d), the Gabor wavelet coefficient is calculated by multiplying brightness values in one block by a wavelet wavelet function.
5. The method of claim 4,
Wherein in the step (d), an average of the wavelet wavelet coefficients calculated using a window centering on the coordinates of all the coordinates of the area represented by each block is extracted as a texture characteristic of each block. An adaptive switching method of a camera.
The method according to claim 1,
Wherein the integrated measure Φ (t; α, β) for the initial image at time t is defined by the following equation (1) in the step (e).
[Equation 1]

Where F _bright (t) and F _gabor (t) are the brightness and texture characteristics, respectively.
The method according to claim 6,
Wherein the step (f) activates the passive camera when the integrated measure is greater than or equal to a predetermined threshold, and activates the active camera if the integrated measure is less than a predetermined threshold.
The method according to claim 1,
Wherein the integrated measure Φ (t; α, β) for the initial image at time t is defined by the following equation (1) in the step (e).
[Equation 1]

Where F _bright (t) and F _gabor (t) are the brightness and texture characteristics, respectively.
9. The method of claim 8,
Wherein in the step (f), the active camera is activated when the integrated scale is greater than or equal to the predetermined threshold value, and the passive camera is activated when the integrated scale is smaller than the predetermined threshold.
A computer-readable recording medium on which a program for performing the adaptive switching method of the hybrid stereoscopic camera according to any one of claims 1 to 9 is recorded.

Images