KR20030076905A - Method and apparatus for a tracking stereo target - Google Patents

Abstract

PURPOSE: A stereo object tracking system and a method thereof are provided to control the convergence angle and the field of view of a stereo camera by using the hierarchical digital image processing and the optic BPEJTC complementarily, thereby realizing the adaptive stereo object tracking in real time. CONSTITUTION: A stereo object tracking method includes the steps of matching backgrounds of a target object between two images by searching a relative motion with respect to the movement speed and direction of a stereo camera via correlation among the input images of continuous three frames by an optic BPEJTC(Binary Phase Extraction Joint Transform Correlator)(201), separating the background and zoning a moving object by extracting target motion components between the two images and making a mask to project to a reference image by a morphology filter(203), and extracting position information of the object to control the convergence angle and the field of view of the camera(205).

Description

Method and system for tracking stereo objects {Method and apparatus for a tracking stereo target}

The present invention relates to a stereo object tracking method and system capable of adaptively tracking a moving target by controlling the viewing angle.

The technique of tracking moving targets in real time has been studied in several applications over the last few decades. Recently, with the development of industrial and military technology, as the demand for unmanned automation system increases, there is a great interest in the technology of tracking moving targets.

In particular, object tracking and remote operation using stereo robot vision technology gives a robot three-dimensional vision like humans, enabling natural work processing as if it is in the field, and compared to conventional two-dimensional vision. It is known to increase the efficiency of remote work by more than 30%. These stereo robot vision systems include stereo camera technology that provides stereoscopic vision, object recognition technology that processes stereo input images to recognize the target object, and continuously tracks and monitors the target object. Stereo object tracking technology.

In general, in a stereo object tracking system based on binocular disparity, when the viewpoints of the target objects do not coincide with the focal planes of the left and right eyes, the target objects appear to overlap in two, thereby increasing the eye fatigue and This results in a decrease in work efficiency. Therefore, when there is stereo disparity in the tracking object, this disparity is eliminated, which is called convergence angle control. That is, in a conventional stereo object tracking system, there is a binocular parallax and a perspective control technique for eliminating stereo parallax with respect to a tracking object, and a camera pan / tilt which always keeps the tracking object centered in the field of view (FOV). Two functions are required: control (hereinafter referred to as FOV control) technology.

In the conventional stereo object tracking system, extraction of a reference image to find the position coordinate of the stereo input image is essential for the control of the gaze angle and the FOV of the tracking object. Most of the region extraction algorithm of the reference image is used to identify the tracking object by removing the background and noise from the relationship between the previous image and the current image using a two-dimensional sequential input image. In other words, a method for extracting a reference image in such a two-dimensional image processing is the image difference method by comparing the front and rear frames, model-based using the characteristics of a predefined model ), Optical flow, and block matching methods have been studied.

However, the differential image method has a problem in detecting a moving target when the image detector is moving. In the model-based method, there is a limitation of difficulty in matching when the input image is relatively deformed or slightly deformed unlike an object of a previously stored model. In addition, since the optical flow method is a pixel-based matching method, it is reported that a large amount of displacement of an image causes a calculation error, which cannot be applied, and matching with a general perspective is difficult. In addition, the block-based method is a method of estimating motion by comparing blocks, and it is difficult to extract moving objects when there is a change in the amount of computation for finding the most suitable block and the background of the before and after frames. .

An object of the present invention is to provide a stereo object tracking method and system that can detect a moving target in real time even when the image detector is moving.

Another object of the present invention is to provide a stereo object tracking method and system that can substantially detect change factors such as changes in relative contrast or shape of a moving target.

Still another object of the present invention is to provide a stereo object tracking method and system that can extract the position information of a moving target easily in real time and then perform gaze control and FOV control using the extracted position information.

1A is a diagram exemplarily illustrating a case where stereo input images of left and right cameras are exactly matched to a gaze point with respect to a tracking object;

FIG. 1B is a diagram exemplarily illustrating a case where two images overlap on a screen due to stereo parallax because a gaze point of a stereo input image of a left and right camera with respect to a tracking object is different.

2 is a schematic representation of the processing flow of an optical-digital stereo object tracking system according to the present invention.

3A and 3B are views exemplarily illustrating l (t + 1) and r (t + 1), which are left and right input images in a t + 1 frame when a camera moves according to an exemplary embodiment of the present invention.

4 is a diagram showing the internal configuration of the optical BPEJTC system for the left image according to an embodiment of the present invention.

5 shows an example of an input plane of an optical BPEJTC according to a preferred embodiment of the present invention.

FIG. 6A shows an existing Convex hull filter consisting of four basic components. FIG.

6B illustrates a modified Convex hull filter consisting of eight components in accordance with one preferred embodiment of the present invention.

7 is a view showing a projection process for the region of the target according to an embodiment of the present invention.

8 is an exemplary view of an optical BPEJTC input plane shown in two stages in accordance with a preferred embodiment of the present invention.

9A and 9B are views illustrating input images when a camera moves according to an exemplary embodiment of the present invention.

10A and 10B illustrate localized results of a target object in accordance with one preferred embodiment of the present invention.

11A and 11B are diagrams showing results of correlation peak values of left and right images shown in a correlation plane according to an exemplary embodiment of the present invention.

<Explanation of symbols for the main parts of the drawings>

201... First step 203. 2nd step

205... 3rd step

In order to achieve the above objects, according to an aspect of the present invention, in the method for adaptively controlling a stereo object tracking system constituting a first image input device and a second image input device, the first image input device Detecting a target object included in the at least two input images through background matching from at least two input images sequentially photographed through the above; separating the target object using a target projection mask; Generating a reference image comprising: extracting first location information by comparing the reference image with at least one target image captured by the second image input apparatus, and using the first location information; Adaptive control method of a stereo object tracking system comprising the step of controlling the image input device Can provide.

The method may further include extracting second location information of the separated target object and controlling the first image input device by using the second location information, wherein the reference image is the separated target object. It is preferable to generate by correcting to correspond to the second position information. The first image input device is preferably pan / tilt controlled using the second position information. The target projection mask may be configured through image differential filtering, AND operation, and morphology filtering. The first location information of the target object may be extracted using an optical Binary Phase Extraction Joint Transform Correlator (BPEJTC). The second image input apparatus may control at least one of the viewing angle and the pan / tilt using the first location information.

According to another aspect of the present invention, in the stereo object tracking system constituting the first image input device and the second image input device, the background matching from at least two input images sequentially captured through the first image input device Means for detecting a target object included in the at least two input images, means for separating the target object using a target projection mask, means for generating a reference image including the target object, and inputting the second image. Means for extracting first position information by comparing the at least one target image captured by the apparatus with the reference image, and means for controlling the second image input apparatus using the first position information; A system can be provided.

According to another aspect of the present invention, a stereo object tracking system constituting a first image input device and a second image input device, the stereo object tracking system includes a memory in which a program is stored and a processor coupled to the memory to execute the program. Wherein the processor detects a target object included in the at least two input images through the background registration from at least two input images sequentially captured by the first image input device by the program, target projection Separating the target object by using a mask, generating a reference image including the target object, comparing the reference image with at least one target image captured by the second image input device, and a first position; Extracting the information and using the first location information; It may provide an adaptive control system of a stereo object tracking system to execute the step of controlling the input device.

Next, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Referring to the stereoscopic object tracking method and system according to the present invention, a moving target is detected through background matching from a sequential input image, and a target projection mask configured using an image difference filter, an AND operation, and a morphology filter is used. Finally, the moving target is localized, and finally, the position information of the moving target is extracted by using optical binary phase extraction joint transform correlator (BPEJTC). Can work In addition, the stereo object tracking method and system according to the present invention presents the possibility of real-time implementation of the adaptive stereo object tracking system through the optical implementation of the moving target object extraction and the camera view control algorithm.

That is, the stereo object tracking method and system according to the present invention sequentially match the background through correlation between two input frames using optical BPEJTC, and then use a moving image target using a modified image differential filter on the front and rear frame images. By detecting the area and applying a modified Convex hull filter, the moving target can be localized. In addition, the stereo object tracking method and system according to the present invention can calculate the position information of the moving object by performing optical BPEJTC between the localized reference image and the stereo input image, finally using the position information of the stereo camera View angle and pan / tilt (FOV) control. Moving angles and pan / tilt (FOV) control of the stereo camera allows moving targets to be tracked in real time. In the present invention, by introducing the Convex hull treatment can reduce the localization error occurring in the existing differential filter, BMA, etc., precisely moving target using the correlation peak that can be phase-correlated in real time through the optical configuration BPEJTC Extraction and use of this method enable the control of perspective and pan / tilt (FOV) of stereo object tracking methods and systems.

In general, in order to match left and right viewpoints with respect to a moving object in an object tracking system using a stereo camera, a stereo parallax of a camera is required when a person tracks a moving object. It should be "0". By doing so, natural observation and improvement of work efficiency without eye fatigue can be achieved. On the other hand, when stereoscopic parallax exists because the viewpoint of the moving object to be tracked does not coincide with the focal planes of the left and right eyes, the moving object is shown as a double, and the eyes are fatigued, thereby greatly reducing work efficiency.

FIG. 1A illustrates an example in which stereo input images of left and right cameras are exactly matched to gaze points for a tracking object, and FIG. 1B illustrates stereo parallax because the gaze points for stereo input images of left and right cameras for tracking objects are different. Is a diagram illustrating an example where two images overlap each other on a screen.

Stereo object tracking method and system according to the present invention is the same as the function of the existing two-dimensional object tracking system in terms of recognizing, extracting and tracking the moving target to be tracked and positioning the moving target in the center of the screen, The difference is that the watch points must be matched and tracked at the same time. In other words, vergence control and pan / tilt (FOV) control of the stereo camera are added. Here, the perspective control is to match the point of view of the stereo camera on the same point (target object) in the three-dimensional space. This point is the point where the stereo parallax becomes "0" so that the observer's left and right eyes are in focus. Can be. In addition, the pan / tilt (FOV) control of the stereo camera controls the pan and tilt (FOV) of the stereo camera as the target object moves so that the tracking object is always in the center of the left and right camera field of view (FOV), such as human eye movement. Refers to the ability to be placed.

According to the present invention, if it is possible to extract the current position information of the tracking object from the input image of the two cameras in the stereo object tracking system, the gaze angle control and the pan / tilt (FOV) control of the camera can be performed at the same time. However, when only stereo parallax for stereoscopic control is extracted in stereo image processing, current position information of an object for tracking must be separately obtained. In addition, according to the present invention, in the case of stereo image processing using the correlation technique, since the current position information of the tracking object can be extracted, it is possible to simultaneously control the viewing angle and pan / tilt (FOV) using the same.

2 is a schematic diagram of a processing flow of an optical-digital stereo object tracking system according to the present invention.

Referring to FIG. 2, the processing flow of the optical-digital stereo object tracking system includes the first step 201 of matching the background of the target object, the background separation and the second step 203 of zoned the target object, and the target object. A third step 205 of extracting the location information is carried out. That is, in the first stage, the optical-digital stereo object tracking system uses the optical BPEJTC to find the relative movement with respect to the moving speed and direction of the camera through the correlation between three consecutive frame input images, and uses the two images. Match the background. The result of matching the background between the two images is used for the background separation process of the second step. In the second step, the optical-digital stereo object tracking system extracts only pure target motion components between two images by applying image difference filter and logical product operation, and constructs a mask using a morphology filter to project the reference image. Area only the moving target object. In the final third step, the optical-digital stereo object tracking system extracts the position information of the moving tracking object by executing the optical BPEJTC between the localized reference image and the stereo input image, and extracts the position information of the moving target object obtained here. Finally, the stereo camera's pan / tilt (FOV) is controlled as well as the stereo camera's perspective control. Each step will be described in more detail with reference to the drawings.

The first step of matching the background of the moving target object will be described in detail as follows.

In general, when the background is identical between sequential input images, the motion of the target object moving with only the image difference filter can be easily detected. However, in the case of a frequently occurring model in which both the background and the target move, the target by background removal Object detection is difficult Therefore, in order to analyze the efficiency of the algorithm proposed in the present invention, a tracking scenario is constructed by dividing the case where the camera displacement is larger than the target displacement and the case where the camera displacement is smaller than the target displacement.

3A and 3B are diagrams exemplarily illustrating l (t + 1) and r (t + 1), which are left and right input images in a t + 1 frame when a camera moves according to an exemplary embodiment of the present invention.

In the example screen in FIG. 3A, the camera displacement is greater than the target displacement, and in the example screen in FIG. 3B, the right side represents the case where the target displacement is larger than the camera displacement.

4 is a diagram showing the internal configuration of the optical BPEJTC system for the left image according to an embodiment of the present invention.

Referring to FIG. 4, between l (t) , which is the left input image of the t frame, and using this, between l (t-1), l (t), or l (t), l (t + 1), which are sequential input images. The correlation between the background and the target object is obtained. The optical BPEJTC system is a phased optical correlator system proposed to eliminate excessive DC or correlation error occurring in the conventional optical JTC system and to cope with the separation condition more flexibly. In other words, in the JTPS (joint transform power spectrum) represented by the light intensity distribution, the autocorrelation component and the cross-correlation component appearing in the same image plane are effectively removed to extract only the phase function, thereby improving the correlation discrimination power and maximizing the light efficiency. In order to configure the binary phase JTPS in the optical BPEJTC system, in addition to the JTPS of the optical JTC, the respective light intensity distributions for the reference image and the target image are required. Fourier transform is performed through the lens 415 to obtain the light intensity of the reference image and the target image through the CCD3 405 and the CCD4 407.

The SLM1 409 image of the optical BPEJTC input plane is Fourier transformed by the first lens 417 to be detected by the CCD1 401 in the form of JTPS. The input plane will be described with reference to FIG. 5.

5 illustrates an example of an input plane of an optical BPEJTC according to an exemplary embodiment of the present invention.

Referring to FIG. 5, the input plane displays l (t-1) frames as reference images and current (ie, l (t) frames as input images) in two stages.

Referring back to FIG. 4, when two images (a reference image and a target image) are simultaneously observed at an arbitrary time, the distance between the images is displayed as the actual moving distance is added to the image interval L. FIG. Equation 1 is a reference image Equation 2 is a target image It is expressed.

here, Is a target included in the image, Is the background that appears identical between two images (frames), and Are background images that are present in the target frame but disappeared in the reference frame due to the shaking of the input device camera.

here, Is the target in the target image (current or input image) Moved by, Is the background of the pan / tilt As shown by each move. Also, Represents a newly input background image in the reference frame due to the movement of the target and the shaking of the camera which is a detector. here, Is a value including W / 2, which is an upper and lower frame interval.

JTPS, which is a light intensity distribution that can be detected by the first CCD camera 401 which is an energy detector by Fourier transforming the two frames configured as described above through the first lens 417, can be represented by Equation 3 below.

Here, L (u, v) represents a Fourier transform formula of l (x, y), and * represents a complex conjugate. Third term in equation (3) And fourth term The BPEJTC correlation output can be finally obtained through the second CCD camera 403 by extracting and binarizing this term from the correlation spectrum between the two frames and inverse Fourier transforming the converted JTPS through the second lens 419. have.

Equation 4 is JTPS reconstructed using Equation 1 and Equation 2. Here, the autocorrelation between the target and the target, the background, and the background is represented by a high correlation peak, and the remaining terms are represented by relatively small values as cross-correlation components generated between different images.

here, Is The Fourier transform of Is The Fourier transform of Is The Fourier transform of Is The Fourier transform of Is The Fourier transform of Is Is the Fourier transform, * is the complex conjugate, Is Phase of Is The phase of each is shown. The autocorrelation between the target and the target, the background and the background is represented by a high correlation peak, and the remaining terms are represented by relatively small values as cross-correlation components between different images.

Accordingly, the final output correlation value c (x, y) may be expressed as in Equation 5.

Here, Edge [·] represents the ideal boundary which generated the phase signal of an image. Δ [·] represents an impulse function. As a result, in the case of a complicated environment, considering the size of the target object, it can be assumed that there are more background components than those of the target object, so the position of the maximum correlation value can be defined as the background, that is, the moving component of the camera. have. here, Denotes correlation and * denotes convolution, respectively. Can be obtained as

The second stage of separating the background and region of the target object will be described in detail as follows.

The input video of three frames in sequence Given by, and obtained via optical BPEJTC The degree of background movement between , In this case, the image difference filter may be used as an easy way to remove the background between the images. The result is represented by equations (6) and (7), which are represented by motion components between the images.

Next, to distinguish common motion from the result of the image difference filter and Targets can be detected by performing AND operations on the values of. Through this process, the motion component is found from three consecutive images. That is, the value of the motion component of the target object Can be expressed as in Equation 8, Means the binarization process.

In the present invention, for efficient binarization, a method of minimizing variance of the entire region is used in a probabilistic manner. That is, in one preferred embodiment of the present invention, in order to binarize the result of image difference, the threshold value τ that minimizes the in-group variance or maximizes the variance between the groups is maximized. By probabilistic selection over the entire range, the variance of the entire image is obtained as in Equation (9). here, Is the probability distribution at a portion less than the threshold τ, Represents a probability distribution value when the threshold value is greater than t. And, Is the average of the first area, Means the average value of the second area. Therefore, the dispersion difference between each region is minimized while the dispersion in each region is minimized. The threshold value is determined by finding t, which is the maximum, and the result is given by Equation 10.

In general, the morphology filter is widely used to remove noise on a background that is not lost in image processing, or to process contour detection, filtering, and thinning. In a preferred embodiment of the present invention, the sequential iteration process of the Convex hull filter and the closing technique is applied to target region in the reference image from the motion detection result. By applying the Convex hull filter and the closing technique throughout the image, the target mask can be created by not only smoothing the boundary but also filling the empty space within each boundary.

On the other hand, the conventional Convex hull filter is composed of four basic components as shown in Figure 6a. However, when such existing components are applied to each image, a surplus outline pattern larger than the actual target is generated, and thus, accurate localization cannot be expected. In order to solve this problem, that is, in order to extract an image as close to the original target object as possible, a modified mask is set in an exemplary embodiment of the present invention. As shown in FIG. 6B, eight modified Convex hull filters are displayed. can do.

Input video A, the component As the initial condition before applying the Convex hull filter In this case, the entire process can be represented by Equations 11 and 12.

Where ø represents a hit-or-miss conversion. Applying the components one by one Is a temporary process for finding a completely convergent result for a component. Will be expressed. Therefore, this is the condition that the newly obtained result X converges to the previous value. CONV inserts a subscript, meaning that the calculation is repeated repeatedly until this is satisfied. As a result, C (A), the final Convex hull filter result, can be obtained by summing all the image results {D} ^ {i} to which all the components are applied. Therefore, in order to set the moving area mask from the detection result of the target object, a new morphology filter combining Convex hull filtering and closing process is additionally applied to Equation 8, thereby applying a target mask as shown in Equation 13. Will be set.

Target image obtained by Equation 13 to localize the position of the target object to the coordinate value in consideration of the recognition process of the rear stage Projecting to will only highlight the target. Here, a method of projecting the detected masks in the x and y directions, respectively, is newly introduced.

7 is a view showing a projection process for the region of the target according to an embodiment of the present invention.

Referring to FIG. 7, the projection process first projects the x-axis from the motion detection value to find the position of the target on the x-axis. After that, by projecting on the y-axis again, the exact position of the target on the y-axis can be found to separate only the target from the background.

Also, the position coordinates of the tracking object in the mask of the left image with the background separated To locate and center the camera screen After moving by, the reference image of the optical BPEJTC input plane is used. And the shift value Becomes the pan / tilt control value of the left camera.

Finally, a third step of extracting the location information of the target object will be described in detail.

Reference image of the target mask to obtain the location information of the target object The relative distance value between two objects is obtained through optical BPEJTC between the image and r (t) on the right side of the image. Here, the procedure of performing the optical BPEJTC used to extract the location information of the target is the same as the background matching process described in the first step, and thus the description thereof will be omitted.

That is, optical BPEJTC input plane The SLM1 image is Fourier transformed by the first lens to be detected by the CCD1 in the form of JTPS. The input plane represented by FIG. 4 may be divided into two stages as shown in FIG. 8 and represented as an optical BPEJTC input plane. Reference image used as optical BPEJTC input in Figure 4 And the position coordinates of the tracking object of the input image [r (t)] Given by In addition, the result obtained in the correlation plane by using an inverse Fourier transform using JTPS modified in binary phase form as an input of SLM2 can be obtained as shown in Equation (14).

Equation (14) is detected as the correlation peak in CCD2, the correlation plane shown in FIG. The peak value can obtain the position coordinates of the peak value by the relative distance between the reference image and the right image in the center coordinates. In addition, the position coordinate value of the tracking object of the right image from the position coordinates And pan / tilt (FOV) control of the right and left cameras Can be obtained respectively.

In Equation 14, Edge [·] represents an ideal boundary extraction function of an image. In Equation 15, it can be seen that Fourier transform removes an amplitude component and inverse Fourier transform to obtain an image boundary extraction result. As a result, optical BPEJTC not only eliminates the correlation error caused by DC and background, but also exhibits excellent peak-to-sidelobe in spatial matched filters.

Therefore, the control value of the left camera obtained from the preprocessing of the l (t) image Control Values of Right Images Obtained by Optical BPEJTC By controlling the pan / tilt (FOV) of the stereo camera, the real-time tracking of the stereo object is enabled.

Next will be described in detail with respect to the experimental process and the result of applying the stereo object tracking method and system according to the present invention.

In this experiment, left and right image input cameras were used by Tokyo Electronics Co., Ltd. CS-8239B camera, and the size of input image data including moving target object (car model) was 256x256 pixels. The frame grabber for image recording was stored using two of Matrox's Metero II / 4 and Metero II MC / 2, and the digital system used Pentium III-800 (256MB).

1. Experiment on the background matching of the target object as the first step

In the scenario for demonstrating the efficiency of the algorithm according to the present invention, the experiment was performed by dividing the case where the camera displacement is larger than the target displacement and the case where the camera displacement is smaller than the target displacement.

9A and 9B are views illustrating input images when a camera moves according to an exemplary embodiment of the present invention.

9A and 9B, the left side of the input image (FIG. 9A) shows the case where the displacement of the camera is greater than the displacement of the target object and the right side (FIG. 9B) shows the case where the displacement of the target object is greater than the displacement of the camera. .

The reference image for background matching is set as l (t) , which is the second frame. Table 1 shows the maximum correlation value position and the camera's moving displacement component obtained by successively correlating the input images sequentially compared to the reference image.

TABLE 1

In Table 1, the correlation peak-to-image position of each image shows that when the camera displacement is larger than the displacement of the target object, the displacement component between images is large, so that the position of the maximum correlation value obtained by correlation using optical BPEJTC is due to the background. have. In addition, the sign of the displacement component is given in a direction that should be moved relatively for background matching between the target image and the reference image. That is, in Table 1, when the camera displacement is larger than the displacement of the target object, the results of correlating the first and second images show negative values in the x direction and positive values in the y direction. This means that when the reference image is the center, it is located on the right side of the reference image in the x direction and on the left side in the y direction. Therefore, in order to match the first image l (t-1) with the background of the second image l (t), if the pixel is moved 16 pixels to the left and five pixels to the bottom, the background between the images becomes identical. Then, by combining the two result values using a binarization process, a logical product operation, etc., only a common moving part can be calculated.

2. An experiment for extracting position information, which is a moving component of a target object, as a second step

After the background is matched through the correlation between the input images, the image is effectively removed by using an image difference filter. FIGS. 10A and 10B illustrate the convex hull filtering and closing technique using the detected motion values of the target. By applying to smooth the boundary line as well as fill in the empty space existing in each boundary surface showing the result of the localized result by mapping the target mask and the result value to the reference image.

10A and 10B, it can be seen that an effective detection of a target object to be tracked and a target mask accordingly are adaptively performed regardless of background noise. In addition, since only a certain size of the localization mask is selected after the Convex hull processing, it can be seen that adaptive target detection is possible regardless of other objects present in the foreground and background.

3. Experiment of extracting the location information of the target object, which is the third step

11A and 11B illustrate the detected and localized left image of the target object when the camera displacement is greater than the target displacement. And a correlation peak value between the input plane for executing the optical BPEJTC and the left and right images appearing in the correlation plane after executing the optical BPEJTC between the current image and the right input image r (t). By using the position coordinate value of the correlation peak, the target object of the left and right images is a distance away from the center of the screen. Can be obtained.

Table 2 shows the moved position coordinates (x, `` y) of the target object finally obtained using the tracking algorithm according to the preferred embodiment of the present invention for the left and right input images of the four frames sequentially. Here, the position coordinates represent positions of the target object moved in the x and y-axis directions based on the center coordinates 128 and 128 of the input image 256x256, respectively.

TABLE 2

In addition, each image at the lower right of FIG. 11 represents a stereo image synthesized after tracking through Table 2, which is a moving component of a target object obtained using the method proposed in the input image of FIG. 9.

Referring to FIG. 11, the target object (car model) is one in which the left and right viewpoints match the target object by precisely controlling the viewing angle and pan / tilt (FOV) of the stereo camera through a tracking algorithm according to the preferred embodiment of the present invention. Although it appears as a composite image of, other foreground and background objects can be seen to overlap two due to stereo parallax.

The present invention is not limited to the above embodiments, and many variations are possible by those skilled in the art within the spirit of the present invention.

According to the present invention, it is possible to provide a novel optical-digital stereo object tracking method and apparatus capable of controlling the viewing angle and pan / tilt of a stereo camera by using a hierarchical digital image processing algorithm and an optical BPEJTC system. have.

In addition, according to the present invention, the possibility of real-time implementation of the adaptive stereo object tracking system can be presented through the optical implementation of the target object extraction and stereo camera perspective control algorithm.

In addition, the present invention can provide a novel hybrid optical-digital stereo object tracking method and apparatus using a hierarchical digital algorithm and optical BPEJTC as a novel approach to stereo object tracking. That is, the proposed object tracking system (device) performs background matching on the images of the target objects which are sequentially input, and then targets the target objects through a target projection mask constructed using an image difference filter, an AND operation, and a morphology filter. After localization, finally, the location information of the target object can be extracted using the optical BPEJTC. In addition, a new optical-digital stereo object tracking method and apparatus (system) capable of real-time tracking of a moving target object by controlling the angle of view and pan / tilt of the camera using the extracted target object position information can be provided. have.

Claims (8)

A method of adaptively controlling a stereo object tracking system constituting a first image input device and a second image input device, Detecting a target object included in the at least two input images through background matching from at least two input images sequentially captured by the first image input apparatus; Separating the target object using a target projection mask; Generating a reference image including the target object; Extracting first location information by comparing the at least one target image photographed by the second image input apparatus with the reference image; And Controlling the second image input apparatus by using the first location information. Adaptive control method of a stereo object tracking system comprising a. The method of claim 1, Extracting second position information of the separated target object; And Controlling the first image input device by using the second location information. Include more, The reference image is generated by correcting the separated target object to correspond to the second position information. Adaptive control method of a stereo object tracking system, characterized in that. The method of claim 2, The first image input apparatus is pan / tilt controlled using the second position information. Adaptive control method of a stereo object tracking system, characterized in that. The method of claim 1, The target projection mask is Configured through image differential filtering, AND operation, and morphological filtering Adaptive control method of a stereo object tracking system, characterized in that. The method of claim 1, The first location information of the target object is Extracted using optical BPEJTC (Binary Phase Extraction Joint Transform Correlator) Adaptive control method of a stereo object tracking system, characterized in that. The method of claim 1, The second image input apparatus is configured to control at least one of gaze angle and pan / tilt using the first position information Adaptive control method of a stereo object tracking system, characterized in that. In the stereo object tracking system constituting the first image input device and the second image input device, Means for detecting a target object included in the at least two input images through background matching from at least two input images sequentially captured by the first image input apparatus; Means for separating the target object using a target projection mask; Means for generating a reference image comprising the target object; Means for extracting first location information by comparing the at least one target image captured by the second image input apparatus with the reference image; And Means for controlling the second image input apparatus using the first location information. Stereo object tracking system having a. In the stereo object tracking system constituting the first image input device and the second image input device, A memory in which a program is stored; A processor coupled to the memory to execute the program Including, The processor by the program, Detecting a target object included in the at least two input images through background matching from at least two input images sequentially captured by the first image input apparatus; Separating the target object using a target projection mask; Generating a reference image including the target object; Extracting first location information by comparing the at least one target image photographed by the second image input apparatus with the reference image; And Controlling the second image input apparatus by using the first location information. Adaptive control system of stereo object tracking system to run.