KR101964433B1 - Target observing method - Google Patents

Abstract

The present invention relates to a target detection and tracking method. The target detection and tracking method includes the following processes: determining whether a target image is an image input after a tracking command from a user when the target image is received; detecting a target by using multiple filters when the target image is input without a tracking command from the user; enabling the user to set a wanted target and make a tracking command; generating a minimum output sum of squared error (MOSSE) filter when the target image is input after a tracking command and if the target image is the first frame after the tracking command; calculating a peak to side-lobe ratio (PSR) if the target image is the second or later frames input after the tracking command from the user and then deriving a final tracking result; comparing the PSR with a threshold value and updating the MOSSE filter and transitioning to an automatic tracking mode if the PSR is greater than the threshold value; and transitioning to a memory tracking mode or a target loss mode in accordance with target tracking at a set memory tracking time if the PSR is less than the threshold value.

Description

Target observing method [

Field of the Invention [0002] The present invention relates to a target observation method, and more particularly, to a target observation method capable of high-speed detection and tracking of an air intrusion target in a complex background.

In order to port embedded devices of real - time high - speed detection and tracking video tracer, we use traditionally less computationally intensive techniques such as center tracking and correlation tracking.

The center tracking method is a performance goal of stably extracting a moving target from a background in a time-varying space according to a basic processing method of calculating a center of a divided target. In order to identify a target and distinguish the target from the background, . In addition, the correlation tracking technique searches for the displacement of the target in the continuous image through block matching every frame using the matching of the reference image (target area) and the search area (the processing area including the target) as a basic processing method, The overall tracking performance largely depends on the improvement of the matching method in the structural aspect rather than the correlation calculation and the technique that effectively reflects the information of the current target on the reference shape

Using this center tracking method and correlation tracking method, it is easy to track the antiaircraft target of the sky background. That is, since the background is relatively monotonous except for the target, high efficiency can be obtained with a small computation time, and the success rate of the tracking can be considerably increased accordingly. However, these tracking techniques have disadvantages in that tracking is not easy when the target enters a complex background such as a forest or a tower. In other words, there is a disadvantage that the center tracking method is sensitive to the background change and has low reliability for the center position of the target, and the correlation tracking method has a disadvantage that the response to the case where the target or background is rapidly changed is weak. Therefore, when tracking a target entering a complex background using these tracking techniques, the result of tracking results in a different location from the intended location.

In order to solve the problem that it is not easy to track the target in such a complicated background, it is necessary to use a technique which takes a long calculation time. However, the computation-intensive techniques can have a negative impact on the performance of real-time embedded devices. For example, applying a technique with a computation time of 30 Hz to a 100 Hz device may degrade the overall performance of the device due to real-time processing delays.

Korean Patent No. 10-1087592

The present invention provides a target observation method capable of optimizing the detection and tracking time of a target.

The present invention provides a target observation method capable of fast real-time tracking of a target even in a complex background.

According to a first aspect of the present invention, there is provided a target detection method, comprising: determining whether a target image is input after a tracking command of a user when the target image is received; Detecting a target using a plurality of filters if the input image is input without a user's tracking command; And setting a target desired by the user and commanding the tracking.

The step of detecting the target may include the steps of obtaining a Gaussian image and a Laplacian image from the input image, determining threshold values of the Gaussian image and the Laplacian image, respectively, and then binarizing the Gaussian image and the Laplacian image, And extracting a boundary line of the target from the binarized image and detecting a center point.

The Laplacian image is obtained from the Gaussian image.

Acquiring a histogram of the Gaussian image and obtaining a Laplacian absolute value image before determining the threshold value of the Gaussian image and the Laplacian image.

The combination of the Gaussian image and the Laplacian image is performed simultaneously.

A method of detecting a target includes the steps of acquiring a Gaussian image and acquiring a Laplacian image from the Gaussian image, acquiring a histogram of the Gaussian image and acquiring a Laplacian absolute value image, and determining a threshold value of the Gaussian image and the Laplacian image A step of binarizing the Gaussian binarization image and a Laplacian binarization image; a step of segmenting and labeling the binarized image to extract a boundary line of the target from the binarized image to detect a center point; And allowing the user to set a desired target.

The process of combining the Gaussian image and the Laplacian image with the binarization and binarization images is performed at the same time.

At least one of the detection processes uses an IPP (Integrated Performance Primitives) library.

A target tracking method according to a second aspect of the present invention includes the steps of: determining whether a target image is input after a tracking command of a user when the target image is received; Generating a MOSSE filter for a first frame after a tracking command if the input image is an input image after a user's tracking command; If the PSR is larger than the threshold value by comparing the PSR with the threshold, calculate the ratio of the peak to side-lobe ratio (PSR) The process of transitioning to automatic tracking mode after updating the MOSSE filter; And transitioning to the memory tracking mode or the target loss mode according to the target tracking of the set tracking time when the PSR is smaller than the threshold.

The step of generating the MOSSE filter includes the steps of acquiring a plurality of feature images, generating a kernel filter of a previous frame and performing fast Fourier transform, and generating a MOSSE filter by cross-correlating a plurality of feature images with a kernel filter ≪ / RTI >

The plurality of feature images may include a Laplacian absolute value image of a 64 × 64 image and a previous frame image in 1D Sobel x direction, 1D Sobel y direction, 1D Sobel x, y direction, .

The MOSSE filter generates cross-correlation between the fast Fourier transform result of the plurality of feature images and the FFT result of the kernel filter.

Wherein the step of calculating the PSR comprises: generating a correlation map by cross-correlating a plurality of feature images with a MOSSE filter of a previous frame; normalizing the correlation map by inverse fast Fourier transform; calculating a peak value of the y coordinate, and calculating a PSR by measuring a noise value of the peak value by searching for a peak value excluding a certain region on the basis of a peak point of the normalized correlation map.

Calculating a PSR after calculating x and y coordinates in one correlation map, calculating xR and yR in a correlation map, calculating xR and yR, And deriving a final tracking result.

The updating of the MOSSE filter may include generating a kernel filter at the center of the tracking result derived from the current frame and then performing fast Fourier transform on the result of the fast Fourier transform of the plurality of feature images and a fast Fourier transform result of the kernel filter And a process of weighting the cross correlation result obtained in the current frame with the existing MOSSE filter to update the MOSSE filter.

The fast Fourier transform and the inverse fast Fourier transform use an IPP (Integrated Performance Primitives) library.

A target detection and tracking method according to a third aspect of the present invention includes a detection process of detecting a target using a plurality of filters in an input image; And a tracking process of extracting a plurality of feature images after the detection process and generating a MOSSE (Minimum Output Sum of Sqarrel Error) filter and tracking the detected target using the MOSSE filter.

Detecting a target by using a plurality of filters if the input image is an input image without a tracking command of the user; And instructing the tracking.

The step of detecting the target may include the steps of obtaining a Gaussian image and a Laplacian image from the input image, determining threshold values of the Gaussian image and the Laplacian image, respectively, and then binarizing the Gaussian image and the Laplacian image, And extracting a boundary line of the target from the binarized image and detecting a center point.

The Laplacian image is obtained from the Gaussian image.

Acquiring a histogram of the Gaussian image and obtaining a Laplacian absolute value image before determining the threshold value of the Gaussian image and the Laplacian image.

The combination of the Gaussian image and the Laplacian image is performed simultaneously.

Determining whether the input image is an input image after a tracking command of a user, acquiring a Gaussian image if the input image is an input image without a tracking command of the user, and acquiring a Laplacian image from the Gaussian image; Acquiring a histogram of a Gaussian image and acquiring a Laplacian absolute value image; determining a threshold value of the Gaussian image and the Laplacian image and binarizing the histogram; combining the Gaussian binarization image and the Laplacian binarization image; Extracting a boundary of the target from the binarized image to detect a center point, displaying a target candidate, setting a target desired by the user, and then instructing the user to trace the target.

The process of combining the Gaussian image and the Laplacian image with the binarization and binarization images is performed at the same time.

At least one of the detection processes uses an IPP (Integrated Performance Primitives) library.

The tracking process includes an automatic tracking process of automatically tracking a target according to a peak to side-lobe ratio (PSR) and a size of a threshold value, and an automatic tracking process of tracking the size of the PSR, And a target loss process for determining that the target is lost when the memory tracking time is exceeded.

The method includes the steps of determining whether an input image is an input image after a user's tracking command, calculating a peak to side-lobe ratio (PSR) Comparing the PSR with a threshold value; automatically tracking the target if the PSR is greater than a threshold value; determining if the target is tracked within a set memory tracking time when the PSR is less than a threshold value; Detecting a Kalman filter predicted value when the target is tracked within the memory tracking time and tracking the target by memory; and determining that the target is lost if the target is not tracked beyond the memory tracking time.

The tracking process includes a process of generating a MOSSE filter in the case of a first frame input after a tracking instruction of the user and a process of calculating a final tracking result after calculating a PSR in a second or more frames inputted after a tracking instruction of the user .

The step of generating the MOSSE filter includes the steps of acquiring a plurality of feature images, generating a kernel filter of a previous frame and performing fast Fourier transform, and generating a MOSSE filter by cross-correlating a plurality of feature images with a kernel filter ≪ / RTI >

Wherein the step of calculating the PSR comprises: generating a correlation map by cross-correlating a plurality of feature images with a MOSSE filter of a previous frame; normalizing the correlation map by inverse fast Fourier transform; calculating a peak value of the y coordinate, and calculating a PSR by measuring a noise value of the peak value by searching for a peak value excluding a certain region on the basis of a peak point of the normalized correlation map.

The step of deriving the final tracking result includes: generating a correlation map by multiplying a plurality of feature images by a PSR; calculating PSR after calculating x and y coordinates in one correlation map; And deriving a final tracking result by calculating subpixels from the values.

Comparing the PSR with a threshold value, and updating the MOSSE filter if the PSR is greater than the threshold value.

The updating of the MOSSE filter may include generating a kernel filter at the center of the tracking result derived from the current frame and then performing fast Fourier transform on the result of the fast Fourier transform of the plurality of feature images and a fast Fourier transform result of the kernel filter And a process of weighting the cross correlation result obtained in the current frame with the existing MOSSE filter to update the MOSSE filter.

The fast Fourier transform and the inverse fast Fourier transform use an IPP (Integrated Performance Primitives) library.

According to a fourth aspect of the present invention, there is provided a target detection and tracking method comprising: determining whether a target image is an input image after a tracking command is received; Detecting a target using a plurality of filters if the input image is input without a user's tracking command; A process in which a user sets a desired target and commands a tracking; Generating a MOSSE filter for a first frame after a tracking command if the input image is an input image after a user's tracking command; Calculating a peak to side-lobe ratio (PSR) of a second or more frames input after a user's tracking command and deriving final tracking results; Comparing the PSR with the threshold value, and if the PSR is greater than the threshold value, updating the MOSSE filter and transiting to the automatic tracking mode; And transitioning to the memory tracking mode or the target loss mode according to the target tracking of the set tracking time when the PSR is smaller than the threshold.

The step of detecting the target may include the steps of obtaining a Gaussian image and a Laplacian image from the input image, determining threshold values of the Gaussian image and the Laplacian image, respectively, and then binarizing the Gaussian image and the Laplacian image, And extracting a boundary line of the target from the binarized image and detecting a center point.

The Laplacian image is obtained from the Gaussian image.

Acquiring a histogram of the Gaussian image and obtaining a Laplacian absolute value image before determining the threshold value of the Gaussian image and the Laplacian image.

The combination of the Gaussian image and the Laplacian image is performed simultaneously.

A method of detecting a target includes the steps of acquiring a Gaussian image and acquiring a Laplacian image from the Gaussian image, acquiring a histogram of the Gaussian image and acquiring a Laplacian absolute value image, and determining a threshold value of the Gaussian image and the Laplacian image A step of binarizing the Gaussian binarization image and a Laplacian binarization image; a step of segmenting and labeling the binarized image to extract a boundary line of the target from the binarized image to detect a center point; And allowing the user to set a desired target.

The process of combining the Gaussian image and the Laplacian image with the binarization and binarization images is performed at the same time.

At least one of the detection processes uses an IPP (Integrated Performance Primitives) library.

The step of generating the MOSSE filter includes the steps of acquiring a plurality of feature images, generating a kernel filter of a previous frame and performing fast Fourier transform, and generating a MOSSE filter by cross-correlating a plurality of feature images with a kernel filter ≪ / RTI >

Wherein the step of calculating the PSR comprises: generating a correlation map by cross-correlating a plurality of feature images with a MOSSE filter of a previous frame; normalizing the correlation map by inverse fast Fourier transform; calculating a peak value of the y coordinate, and calculating a PSR by measuring a noise value of the peak value by searching for a peak value excluding a certain region on the basis of a peak point of the normalized correlation map.

Calculating a PSR after calculating x and y coordinates in one correlation map, calculating xR and yR in a correlation map, calculating xR and yR, And deriving a final tracking result.

The updating of the MOSSE filter may include generating a kernel filter at the center of the tracking result derived from the current frame and then performing fast Fourier transform on the result of the fast Fourier transform of the plurality of feature images and a fast Fourier transform result of the kernel filter And a process of weighting the cross correlation result obtained in the current frame with the existing MOSSE filter to update the MOSSE filter.

The fast Fourier transform and the inverse fast Fourier transform use an IPP (Integrated Performance Primitives) library.

A target detection and tracking method according to embodiments of the present invention includes a detection process of detecting a target using a plurality of filters in an input image, a detection process of extracting a plurality of feature images after a detection process, a MOSSE (Minimum Output Sum of Sqarrel Error) And tracking the detected target using the MOSSE filter.

Since at least one process of the target detection process and at least one process of the target tracking process are implemented using an IPP (interrupted poison process) library, the present invention can be used in an embedded environment supporting the IPP library. In addition, the computation time of detection and tracking can be optimized, so that it can be applied in environments requiring high-speed tracking. And, it is relatively easy to track in a complex background, so it can be used in an environment that requires tracking in a complicated background.

1 is a flow chart for explaining a target detection and tracking method according to an embodiment of the present invention;
2 is a flowchart illustrating a method of detecting a target according to an embodiment of the present invention.
FIG. 3 and FIG. 4 are flowcharts for explaining a target tracking method according to an embodiment of the present invention; FIG.
FIG. 5 to FIG. 12 are photographs of images in respective processes of the target detection method according to an embodiment of the present invention. FIG.
13 to 17 are photographs of images in each process of the target tracking method according to an embodiment of the present invention.
18 is a configuration diagram of a target detection and tracking apparatus according to an embodiment of the present invention;
19 and 20 are block diagrams of a target detection unit and a target tracking unit of a target detection and tracking apparatus according to an embodiment of the present invention, respectively.
Figs. 21 to 26 are photographs of traces of each frame in one test example of the present invention. Fig.
FIG. 27 is a graph for explaining the detection and tracking execution time of a test example of the present invention. FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be understood, however, that the invention is not limited to the disclosed embodiments, but is capable of other various forms of implementation, and that these embodiments are provided so that this disclosure will be thorough and complete, It is provided to let you know completely.

First, a method of detecting and tracking a target according to an embodiment of the present invention will be schematically described as follows.

The present invention provides a tracking method for detecting a target using a plurality of spatial filters, a tracking process for extracting a plurality of characteristic images and tracking a detected target using a MOSSE (Minimum Output Sum of Sqarrel Error) . ≪ / RTI > In addition, the tracking process can be divided into an automatic tracking process for automatically tracking the target according to the threshold value of the peak to side-lobe ratio (PSR) and a memory tracking process for tracking according to the memory, And the target loss process in which the target is judged to be lost according to the memory tracking time. That is, the tracking process may include an automatic tracking process, a memory tracking process, and a target loss process depending on the PSR and the storage time.

On the other hand, the detection process uses a plurality of spatial filters, and the spatial filter is speeded up by using the parallelization technique and the IPP (Integrated Performance Primitives) library simultaneously. Based on the spatial filter used, the coordinates of the target to be finally detected are calculated through two binarization processes. In the detection process, the user transits to the tracking process at the start of tracking.

In the tracking process, five feature image extraction, Fast Fourier Transform (FFT) which transforms the time base of the MOSSE filter into a frequency base, and Inverse Fast Fourier Transform (IFFT) ) Was accelerated using an IPP library. The PSR of each weighted sum is derived by normalizing the PSR (Peak to Side-lobe Ratio) using each of the five MOSSE filter results, and finally determining the final tracking point. Finally, when the final PSR value exceeds the threshold value, the filter is cumulatively updated from the past to the present in the frame.

On the other hand, the present invention can speed up detection and tracking by using the IPP library. IPP is a cross-platform software library that provides a programming interface that allows users to write highly optimized applications to maximize processor performance. Users can access advanced processor features without writing processor-specific code. This simplifies application optimization, saves development and maintenance time, and enables faster computation times.

The target detection and tracking method according to the present invention will be described in more detail with reference to the drawings.

1 is a flowchart illustrating a method of detecting and tracking a target according to an embodiment of the present invention. FIG. 2 is a flowchart for explaining a target detection method according to an embodiment of the present invention. FIGS. 3 and 4 are flowcharts for explaining a target tracking method according to an embodiment of the present invention. 3 and 4 are flowcharts for explaining a tracking process and an updating process of the target tracking method, respectively.

A method of detecting and tracking a target according to an embodiment of the present invention will be described with reference to FIG. First, when an image of the air target is received (S11), it is determined whether it is a detection process or a tracking process (S12). At this time, whether the detection process or the tracking process is performed can be determined according to a tracking command of the user. That is, if the image is received after the user's tracking command, it enters the tracking process and enters the detection process if the image is received without the user's tracking command.

If it is determined that the image is input without the tracking command of the user and the detection process is performed, a process of detecting the target using a plurality of filters (S13) is performed. For example, a Gaussian image and a Laplacian image are acquired using a Gaussian filter and a Laplacian filter, respectively, and a process of binarizing the images and combining the binarized results is performed.

After detecting the detection point (S14), the user sets a desired target and presses the start tracking button to enter the tracking process (S15). At this time, if the image is binarized, the center point of each target becomes unclear. Therefore, segmentation and labeling are performed to extract the boundary of the target from the binarized image to detect the center point, and apply the Kalman filter to the center point component. Then, the target that has been detected continuously among the targets input by the Kalman filter is displayed as a target candidate, the user sets a desired target, and then the tracking start button is clicked to enter the tracking process.

If it is determined that the input image is a tracking process after the tracking command of the user, the process of generating the MOSSE filter (S17) is performed in the case of the first frame after the tracking command (S16). A plurality of feature images are acquired to perform a fast Fourier transform (FFT) to generate a MOSSE filter, a kernel filter is generated at a center of a previous frame to perform a fast Fourier transform, And cross-correlates the FFT result of the kernel filter to generate a plurality of MOSSE filters. That is, the MOSSE filter is generated in the same number as the number of feature images.

In step S16, a peak-to-side-lobe ratio (PSR) is calculated in step S18 and a subpixel is calculated in step S19. PSR can be calculated after cross-correlation of the FFT result of the plurality of feature images and the MOSSE filter of the previous frame and performing inverse fast Fourier transform (IFFT) to generate a correlation map, normalize and calculate the peak value of the x and y coordinates . In addition, the subpixel may be multiplied by the PSR of the five feature images to generate one correlation map, and then one correlation map may be used to calculate the x and y coordinates, calculate the PSR, and calculate the subpixels from the x, y peak values .

Next, the PSR is compared with the threshold value (S20). If the PSR is larger than the threshold value, the MOSSE filter is updated (S21) and then the automatic tracking mode is entered (S22). That is, if the PSR exceeds a certain threshold value, it is judged that the tracking is good, and in this case, the MOSSE filter update is performed.

However, if the PSR is less than the threshold value (S20), the transition is made to the memory tracking mode or the target loss mode. That is, if the target is traced within a predetermined time, for example, within 3 seconds (S23), the Kalman filter predicted value is detected (S24) and transits to the memory tracking mode (S25). If the target is lost for 3 seconds or more, (S26). On the other hand, if the target is found again within 3 seconds after the memory tracking mode transition, the automatic tracking mode is transited again. Otherwise, the target loss mode is transited.

A target detection method according to an embodiment of the present invention will be described in more detail with reference to the drawings. FIG. 2 is a flow chart of a target detection method according to an embodiment of the present invention, and FIGS. 5 to 12 are image photographs of each step of the target detection method.

A detection method according to an embodiment of the present invention includes a process of obtaining a Gaussian image and a Laplacian image (S111 and S112), a process of binarizing a Laplacian image (S113, S114, and S115), a process of binarizing a Gaussian image (S116 and S117 , S118), a process of combining the Laplacian binarization image and the Gaussian binarization image (S119), a segmentation and labeling process (S120), a detection point detection process (S121), and a process of transition to a tracking process (S122) .

First, an image is acquired using a multi-filter. After acquiring a Gaussian image (S111), a Laplacian image is obtained (S112). For example, a 3 × 3 Gaussian image is acquired using a Gaussian filter, and a 5 × 5 Laplacian image is acquired using a Laplacian filter. At this time, the Laplacian image can be acquired after acquiring the Gaussian image. That is, the Laplacian image can be obtained by performing Laplacian filtering on the Gaussian image obtained by Gaussian filtering. 6, the Gaussian image from which noise has been removed can be obtained from the original image as shown in FIG. 5 by Gaussian filtering, and the Laplacian image Images can be acquired.

Subsequently, a Laplacian absolute value image is acquired (S113), a threshold value of the Laplacian absolute value image is determined (S114), and the binarized image is binarized (S115). In the input image, there may be a dark target as well as a bright target than the average brightness of the background, so not only a positive value by a bright target but also a negative value by a dark target may be displayed. Therefore, the Laplacian absolute value is taken, converted to a positive number, and binarized using the mean and standard deviation of the Laplacian absolute value image. The binarization process is performed with a value greater than the threshold value set to 1 and a smaller value set to 0. On the other hand, the Laplacian absolute value image is acquired from the Laplacian image and can be obtained as shown in FIG.

Next, a histogram of the Gaussian image is acquired (S116), a threshold value of the Gaussian image is determined (S117), and binarized (S118). The binarization of the Gaussian image is performed by setting a value larger than the threshold value to 1 and a smaller value to 0, as in the case of binarization of the Laplacian image. Meanwhile, the Laplacian absolute value image acquisition, the threshold value determination and binarization process (S113, S114, S115), the Gaussian image histogram RAM acquisition, the threshold value determination and the binarization process (S116, S117, S118) can be performed at the same time. That is, the binarization process of the Laplacian image and the binarization process of the Gaussian image can be performed in parallel. Of course, the binarization process of the Gaussian image may be performed after the binarization process of the Gaussian image. However, it is possible to speed up the computation by performing the parallelization process of the Laplacian image and the Gaussian image in parallel.

Next, the Laplacian absolute value binarization image and the Gaussian absolute value binarization image are combined (S119). When two binarization images are combined, an image as shown in FIG. 10 is obtained. If the Gaussian image is binarized, binarization can be done well for the large image, but binarization for the complex background such as tower and mountain is not performed well. On the other hand, binarization of the Laplacian image is not good for the large image but binarization for the complex image is good. Therefore, we combine Gaussian binary images and Laplacian binarization images to complement the disadvantages of Gaussian binarization and Laplacian binarization. At this time, Laplacian binarization, Gaussian binarization, and coupling can be performed at the same time. That is, 0 and 1 by Laplacian binarization and 0 and 1 by Gaussian binarization can be OR'ed to combine the Laplacian binarization image with the Gaussian binarization image.

Subsequently, segmentation and labeling are performed, and a Kalman filter is applied to the detected target candidates (S120). When the image is binarized, the center of each target of the binarized image becomes unclear. Thus, segmentation and labeling are performed to extract the boundary of the target from the binarized image as shown in FIG. 11 to detect the center point. In addition, the Kalman filter can be applied with the component of the center point to predict the target position of the next frame.

Then, only the target that has been continuously detected among the targets input by the Kalman filter is displayed as the target candidate, and the user can select the desired target (S121). As shown in FIG. 12, after the user sets a desired target, a tracking start button is clicked to enter a tracking process (S122).

The target tracking method of the present invention tracks a target detected by a detection process. The target tracking method includes a tracking process of generating a MOSSE filter and calculating a subpixel after a peak to side-lobe ratio (PSR) And updating the MOSSE filter if the peak to side-lobe ratio (PSR) is greater than or equal to the threshold value. The tracking process is shown in FIG. 3, and the updating process of the MOSSE filter is shown in FIG. 13 to 17 are photographs of images in each step of the target tracking method according to an embodiment of the present invention.

Referring to FIG. 3, the tracking process of the target tracking method according to the present invention includes a process of obtaining a plurality of feature images (S211), a process of generating a kernel filter in the case of a first frame to be tracked (S212), and performing a fast Fourier transform (S214) for performing a cross correlation between the feature image and the kernel filter (S214), generating a MOSSE filter (S215), and if not for the first frame to be traced (S212) A step S216 of performing a cross correlation of the filter, a step of normalizing the correlation S217, a step of detecting a peak point of each map S218, a step of calculating a PSR S219, A process of generating a map (S220), a process of calculating a final PSR and calculating a subpixel value (S221). That is, in the tracking process, a plurality of feature images are acquired to generate an initial MOSSE filter, a plurality of correlation maps (IFFT images) are combined into one map, and a maximum point of x and y coordinates is found Calculate subpixels. Further, in the case of the first frame to be traced, a first MOSSE filter is generated by cross correlation between FFT images of a plurality of features and an FFT image of a kernel filter, and thereafter, Frame Features After cross correlation of FFT images, IFFT process is performed to obtain a correlation map, correlation map normalization is performed, and one correlation map is generated to obtain a maximum point of x, y coordinates And calculates a subpixel.

The target tracking process of the present invention will be described in more detail as follows.

S211: Obtains a plurality of feature images. Acquires a predetermined image in a previous frame to acquire a plurality of feature images, and acquires a Laplacian absolute value image from the acquired image. An image of the previous frame can acquire a 64x64 image at the center of the previous frame, and takes a Laplacian absolute value and converts it to a positive number. Subsequently, 1D Sobel x direction and 1D Sobel y direction image are obtained from 64x64 image of the previous frame center, and 1D Sobel x, y direction image is obtained. Thus, five feature images can be obtained. That is, a 64 × 64 image (original image) and a Laplacian absolute value image of a previous frame can be acquired, respectively, in 1D Sobel x direction, 1D Sobel y direction, 1D Sobel x, y direction, . FIG. 13 shows these five feature images. 13 (a) through 13 (e) show images in a 1D Sobel x direction image (FIG. 13A), a 1D Sobel y direction image (FIG. 13B) 13 (c)), a 64 × 64 image (original image) image of the previous frame (FIG. 13 (d)), and a Laplacian absolute value image (FIG. 13 (a) to 13 (e) show images of the respective steps in the drones as targets.

Each of the five feature images is multiplied by a cosine window, and a fast Fourier transform (FFT) is performed on the cosine window multiplied image. That is, in order to prevent noise from being introduced by the side lobe, the five feature images are multiplied by the cosine window, respectively, and subjected to fast Fourier transform to convert the time base to the frequency base. At this time, the FFT for each image can be speeded up by using the IPP library.

S212: After acquiring the plurality of feature images in this way, it is confirmed whether or not the first frame is traced. That is, in the detection method, the target is detected and the trace start command is issued by pressing the start trace button, and it is confirmed that the first frame is inputted.

S213: In the case of the first frame input after the tracking command, a kernel filter of the previous frame is generated and subjected to fast Fourier transform (FFT). That is, a kernel filter is generated from the center of the previous frame, for example, 64x64, and the generated kernel filter is subjected to fast Fourier transform. 14 shows an image of a dron as an air target generated by a kernel filter.

S214: Cross correlation is performed on the FFT result of the five feature images and the FFT result (S213) of the kernel filter. That is, the FFT result of the five images generated in step S221 and the FFT result of the kernel filter generated in step S213 are cross-correlated, respectively.

S215: A plurality of MOSSE filters are initially generated by cross-correlation between the feature image and the kernel filter. That is, five MOSSE filters are generated according to the number of feature images. 15 (a) to 15 (d) are images of a MOSSE filter. 15 (a) is an image of a MOSSE filter generated by a cross correlation between a 1D Sobel x-direction image and a kernel filter, FIG. 15 (b) is a view of a 1D Sobel y- 15 (c) is an image of a MOSSE filter generated by cross correlation of a 1D Sobel x, y direction image and a kernel filter, and FIG. 15 (d) FIG. 15 (e) shows an MOSSE filter image generated by the cross correlation between the 64 × 64 image (original image) image of the previous frame and the kernel filter. FIG. Respectively.

S216: If the frame is not the first frame to be inputted after the tracking command (S220), that is, if the frame is the second frame or more after the tracking command, a correlation map is generated by cross-correlating the feature image and the MOSSE filter. That is, the result of the FFT of the five feature images is cross-correlated with the MOSSE filter of the previous frame, respectively. Thus, five correlation maps are generated as shown in FIG. 16 (a) is a correlation map image obtained by cross-correlation between a 1D Sobel x-direction image and the MOSSE filter of FIG. 15 (a), FIG. 16 (b) is a 1D Sobel y- 16C is a correlation map image obtained by cross-correlation of the 1D Sobel x, y direction image and the MOSSE filter of FIG. 15 (c), and FIG. 16 (d) is a correlation map image obtained by cross-correlation between the 64 × 64 image (original image) image of the previous frame and the MOSSE filter of FIG. 15 (d), and FIG. 16 (e) 15 (e) shows the correlation map image by the cross-correlation of the MOSSE filter.

S217: The cross correlation between the feature image and the MOSSE filter is normalized. To do this, inverse fast Fourier transform (IFFT) is performed on each cross correlation result, and the correlation map is normalized. That is, IFFT is performed on each of the five cross-correlation results, and then each correlation map is normalized. At this time, the IFFT also applies the IPP library for speedup.

S218: The peak point of each map is detected. That is, the peak values of the x and y coordinates in the normalized five correlation maps.

S219: Calculate the peak to side-lobe ratio (PSR). That is, the PSR is calculated after calculating the peak value in the normalized correlation map. For this purpose, a peak to side-lobe ratio (PSR) is calculated by measuring a peak value in a region except 11 × 11 from the peak point reference for each map and measuring the noise level relative to the peak. The formula for calculating this PSR is as follows.

Here, Corrpeak is a value derived from the result of step S216. That is, Corrpeak means to find the maximum value among the 64x64 image values derived from the process S216. Here, 0 ≤ x <WIDTH means 0th to 63rd pixels in the x direction and 0 ≤ x <HEIGHT means 0th to 63rd pixels in the y direction. Therefore, since the result image is 64 × 64, the values of WIDTH and HEIGHT are 64, respectively. On the other hand, i of Corrpeak means the order of acquiring each feature (1D sobel x direction, 1D sobel y direction, 1D sobel x, y direction, original image, LOG absolute value).

In addition, SidelobePeak is a process of finding the maximum value at points other than the 11 × 11 image centered on the x and y points of Corrpeak. x <Corrpeak _x -11 and x> Corrpeak _x +11 is the portion excluding the eleventh region in the x direction and y <Corrpeak _y -11 and y> Corrpeak _y +11 is the portion excluding the eleven region in the y direction to be. SidelobePeak side i means the order of obtaining each feature (1D sobel x direction, 1D sobel y direction, 1F sobel x, y direction, original image, LOG absolute value).

The PSR divides the values of Corrpeak and SidelobePeak. That is, the ratio of the maximum value obtained from the 11 × 11 outer region in the x and y directions with respect to the value of the center point is obtained. This is to see if the peak value is several times brighter than the surrounding peak value. PSR side i means the order of acquiring each feature (1D sobel x direction, 1D sobel y direction, 1D sobel x, y direction, original image, LOG absolute value).

S220: One correlation map is generated from a plurality of correlation maps. Normalization of the five PSRs is performed, and then the normalized PSRs are multiplied to generate one correlation map as shown in FIG. That is, a plurality of calculated PSRs are normalized once again at a full size, and a correlation map normalized by each of the plurality of PSRs is multiplied by a normalized PSR, and the resultant values are all added to generate one correlation map. The formula for generating one correlation map is as follows.

PSR_norm is a value obtained by dividing the PSR value obtained by the total PSR sum. That is, each PSR is normalized. i is the order of acquiring each feature (1D sobel x direction, 1D sobel y direction, 1D sobel x, y direction, original image, LOG absolute value).

The Corr _mix combines the respective correlation maps derived from the process S216 into one. That is, the result of the process S220. The PSR_norm is multiplied by the correlation map obtained from the process S216 and added together. That is, the correlation weight is multiplied by the normalized PSR from each correlation map to determine the weight of the corresponding index, and these values are summed together to form one correlation map.

S221: The final PSR is calculated and the subpixel value is calculated. In one correlation map, the peak values of the x and y coordinates are calculated and normalization of the correlation map is performed. After calculating the final PSR, the subpixel is calculated based on the x and y values obtained from the correlation map normalization. The final PSR is calculated by finding a peak point with one correlation map finally obtained, and the subpixel value is calculated based on the peak point to derive the final tracking result.

The process of updating the MOSSE filter in the tracking process will be described with reference to FIG.

It is judged that the tracking is performed when the PSR derived from one correlation map exceeds a specific threshold value. That is, as shown in step S20 of FIG. 1, when the PSR exceeds the threshold value, the MOSSE filter update is performed. That is, when the PSR is larger than the threshold value, the update of the MOSSE filter is performed, and the updating process of the MOSSE filter is described as follows. On the other hand, when the PSR is smaller than the threshold value, it enters the memory tracking mode or the target loss mode as described with reference to FIG.

First, a 2? Kernel filter is newly generated around a tracking result point derived from the current frame (S301), and a fast Fourier transform (FFT) is performed (S302). Next, the five feature FFT images and the kernel filter FFT images are cross-correlated with each other (S303). Then, the MOSSE filter is updated by weighted summing the cross correlation results obtained from the existing MOSSE filter and the current frame (S304). Then, as described with reference to Fig. 1, the automatic tracking mode is entered. The formula for updating the MOSSE filter by weighted sum is as follows.

When the kernel filter is created and the FFT is performed, the real part and the imaginary part appear. The real part is expressed as real, and the imaginary part is expressed as image, and expressed as Kernal _real and Kernal _image . This corresponds to the processes S213, S301 and S302. This is an attempt by the user to intervene to stabilize the tracking performance. The Kernel filter is generated by calculating the difference of position in the x and y direction from the current target center, calculating it in 2 sigma form, and multiplying it with each other to generate bidirectional filter. The formula is as follows.

Where i is the position of the horizontal or vertical component and peak is the position of the peak point. That is, x and y components are obtained using the above equations and finally multiplied. The FFT of these formulas yields the real and imaginary components of the kernel filter.

feature _real and featurel _image are values obtained by real and imaginary FFT of features (1D sobel x direction, 1D sobel y direction, 1D sobel x, y direction, original image, LOG absolute value).

CrossCor _real , and CrossCor _image are values obtained by performing steps S221 and S303 using Kernal _real , Kernal _image , feature _real , and feature _image .

MOSSE _real and MOSSE _image are the process of updating MOSSE filter using Kernal _real , Kernal _image , feature _real , featurel _image , and CrossCor _real and CrossCor _image obtained above. The Learning_rate is set to the optimum value in the experiment and the MOSSE filter is updated by accumulating the current filter value while maintaining the value of the past MOSSE filter according to the Learning_rate ratio.

18 is a block diagram illustrating a configuration of a target detection and tracking apparatus according to an embodiment of the present invention. FIG. 19 is a block diagram for explaining a configuration of a target detection unit of a target detection and tracking apparatus according to an embodiment of the present invention, and FIG. 20 is a block diagram for explaining the configuration of a target tracking unit.

Referring to FIG. 18, the target detection and tracking apparatus according to an embodiment of the present invention may include a target detection unit 1000 and a target tracking unit 2000. Further, it may further include an input unit 3000, a control unit 4000, and a memory unit 5000. 19, the target detection unit 1000 includes a target modeling unit 1100 that removes noise from an input image and emphasizes an edge, and a target modeling unit 1100 that uses an absolute A target region extracting unit 1300 for extracting a target region by binarizing an image in which a cut-out value and a threshold value are determined, and a target region extracting unit 1300 for detecting a center point of the binarized image, And a target selection unit 1500 for allowing a user to select a target from the target candidates. 20, the target tracking unit 2000 includes a feature image acquiring unit 2100 for acquiring a plurality of feature images, a fast Fourier transform unit 2100 for converting a time-based signal into a frequency- A determination unit 2300 for determining whether the image input after the tracking command is the first frame and whether the PSR exceeds a predetermined threshold value, and a determination unit 2300 for determining whether a plurality of images including a kernel filter and a MOSSE filter A map generator 2500 for generating a plurality of correlation maps by cross-correlating the feature images with the MOSSE filters and generating a correlation map from the plurality of correlation maps, An inverse fast Fourier transform unit 2600 for performing an inverse fast Fourier transform to convert a signal based on a signal to a time base signal and an operation unit 2700 for detecting a peak point of each map and calculating a PSR,It can hamhal.

The target detection unit 1000 of the target detection and tracking apparatus according to an embodiment of the present invention will be described in detail with reference to FIG.

The target modeling unit 1100 removes the noise of the image input through the input unit 3000 and makes the edge components stand out. That is, the target modeling unit 1100 removes the noises of the input image and highlights the edge components. Therefore, the target modeling unit 1100 executes a preliminary step for target candidate detection. For this, the target modeling unit 1100 may include a plurality of filters. For example, the target modeling unit 1100 may include a Gaussian filter for Gaussian filtering the input image to obtain a Gaussian image, and a Laplacian filter for filtering the Gaussian filtered image. By filtering the input image using the Gaussian filter, it is possible to acquire the Gaussian image from which the noise of the input image is removed. By filtering the Gaussian filter image using the Laplacian filter, the Laplacian image with the edge component can be obtained.

The preprocessing unit 1200 preprocesses the Gaussian image and the Laplacian image before binarization. That is, the preprocessor 1200 obtains the Laplacian absolute value and determines a threshold value of the Laplacian absolute value image. Also, the preprocessing unit 1200 obtains a histogram of the Gaussian image and determines a threshold value of the Gaussian image. The pre-processing unit 1200 determines the threshold value of the Gaussian image and the threshold value of the Laplacian image. At this time, the Laplacian absolute value and the threshold value determination, the Gaussian image histogram acquisition, and the threshold value determination can be performed at the same time. That is, a first preprocessing means for determining a Laplacian absolute value and a threshold value, and a second preprocessing means for acquiring a Gaussian image histogram and determining a threshold value are provided separately so that the first and second preprocessing means simultaneously determine the Laplacian and Gaussian threshold values . Of course, either Laplacian or Gaussian may proceed first and the other may proceed later. However, the threshold values of Laplacian and Gaussian can be determined simultaneously for fast computation speed.

The target region extracting unit 1300 binarizes the Gaussian image and the Laplacian image having the threshold value determined by the preprocessing unit 1200 to extract the target area. That is, the target area extracting unit 1300 performs binarization by setting a value greater than a threshold value of the Gaussian image and the Laplacian image to 1 and setting a smaller value to 0. [ At this time, binarization of Laplacian and Gaussian can proceed simultaneously. That is, a means for binarizing the Laplacian image and a means for binarizing the Gaussian image are provided, so that the binarization of Laplacian and Gaussian can be performed at the same time. Of course, binarization of the Gaussian image may be followed by binarization of the Gaussian image. However, the computation speed can be increased by simultaneously performing the binarization of the Laplacian image and the binarization of the Gaussian image. In addition, the target region extracting unit 1300 may combine binary images of Gaussian and Laplacian. That is, a combining unit is provided to combine the Gaussian binarization image and the Laplacian binarization image. If the Gaussian image is binarized, binarization can be done well for the large image, but binarization for the complex background such as tower and mountain is not performed well. On the other hand, binarization of the Laplacian image is not good for the large image but binarization for the complex image is good. Therefore, we combine Gaussian binary images and Laplacian binarization images to complement the disadvantages of Gaussian binarization and Laplacian binarization. Thus, the edge can be emphasized and the target area can be extracted. On the other hand, Laplacian binarization, Gaussian binarization, and coupling can be performed at the same time. That is, 0 and 1 by Laplacian binarization and 0 and 1 by Gaussian binarization can be OR'ed to combine the Laplacian binarization image with the Gaussian binarization image.

The target candidate determining unit 1400 detects a center point of the binarized image and determines a target candidate. That is, since the center point of each target is not clear, the binarized image is segmented and labeled to extract the boundary of the target from the binarized image to detect the center point. In addition, the target position of the next frame can be predicted by applying a Kalman filter to an image in which the center point is detected by segmentation and labeling.

The target selecting unit 1500 displays only the target that has been continuously detected among the targets input by the Kalman filter as a target candidate so that the user can select the desired target. Therefore, after the user sets a desired target, the tracking process can be started by pressing the start tracking button.

The target tracking unit 2000 of the target detection and tracking apparatus according to an embodiment of the present invention will be described in more detail with reference to FIG.

The feature image acquiring unit 2100 acquires a plurality of feature images. Acquires a predetermined image in a previous frame to acquire a plurality of feature images, and acquires a Laplacian absolute value image from the acquired image. The image of the previous frame can acquire a 64x64 image at the center of the previous frame, and can take a Laplacian absolute value and convert it to a positive number. Also, 1D Sobel x direction and 1D Sobel y direction images are obtained from the 64x64 image of the previous frame center, and 1D Sobel x and y direction images are obtained. Accordingly, the feature image acquisition unit 2100 can acquire five feature images. That is, a 64 × 64 image (original image) and a Laplacian absolute value image of a previous frame can be obtained, respectively, in 1D Sobel x direction, 1D Sobel y direction, 1D Sobel x, y direction, .

The fast Fourier transformer 2200 performs fast Fourier transform to convert a time-based signal into a frequency-based signal. The fast Fourier transform unit 2200 performs fast Fourier transform on five feature images. Each of the five feature images is multiplied by a cosine window, and fast Fourier transform is performed on an image multiplied by a cosine window. That is, in order to prevent noise from being introduced by the side lobe, the five feature images are multiplied by the cosine window, respectively, and subjected to fast Fourier transform to convert the time base to the frequency base. At this time, the FFT for each image can be speeded up by using the IPP library. In addition, the fast Fourier transform unit 2200 performs fast Fourier transform on the kernel filter generated in the case of the first frame input after the tracking command. That is, the kernel filter is generated for the first frame after the tracking command, and the fast Fourier transform unit 2200 performs fast Fourier transform on the kernel filter. The fast Fourier transform unit 2200 may perform a fast Fourier transform on a newly generated 2? Kernel filter centered on the tracking result point derived from the current frame. That is, the fast Fourier transform unit 2200 performs fast Fourier transform on the newly generated 2? Kernel filter as a process for updating the MOSSE filter. Therefore, the fast Fourier transform unit 2200 performs fast Fourier transform on the five feature images and the kernel filter.

The determination unit 2300 determines whether the image input after the tracking command is the first frame. That is, after acquiring a plurality of feature images, the target is detected in the detection method, and a tracking command is issued by pressing the start tracking button, and it is confirmed whether the first frame is inputted. According to the determination of the determination unit 2300, the filter generation unit 2400 may generate the MOSSE filter or calculate the PSR to calculate the subpixel. In addition, the determination unit 2300 determines whether the PSR exceeds a specific threshold value. If the PSR exceeds the specific threshold value according to the determination result of the determination unit 2300, it is determined that the tracking is well performed and the MOSSE update is performed. That is, when the PSR is larger than the threshold value, the MOSSE filter is updated. Meanwhile, the determination unit 2300 may determine whether to operate in an automatic tracking mode, a memory tracking mode, or a target loss mode. That is, when the PSR is greater than the threshold value, the determination unit 2300 performs the update of the MOSSE filter and then transits to the automatic tracking mode. However, when the PSR is less than the threshold value, the transition is made to the memory tracking mode or the target loss mode. That is, when the target is traced within a predetermined time, for example, within 3 seconds, the Kalman filter predicted value is detected and transited to the memory tracking mode. If the target is lost for 3 seconds or more, the target loss mode is transited. On the other hand, if the target is found again within 3 seconds after the memory tracking mode transition, the automatic tracking mode is transited again. Otherwise, the target loss mode is transited.

The filter generation unit 2400 may include a kernel filter generation unit and a MOSSE filter generation unit. The kernel filter generator generates a kernel filter of the previous frame in case of the first frame input after the tracking command. That is, the kernel filter generator generates a kernel filter from, for example, 64x64 at the center of the previous frame. Also, the kernel filter generator can newly generate a 2? Kernel filter around the tracking result point derived from the current frame. That is, if it is determined by the determination unit 2300 that the PSR exceeds the threshold value, the filter generation unit 2400 generates a new kernel filter in order to update the MOSSE filter. The MOSSE filter generator generates a plurality of MOSSE filters by performing cross correlation on the FFT result of the five feature images and the FFT result of the kernel filter. That is, a plurality of MOSSE filters are first generated by cross-correlating the FFT result of the five feature images and the FFT result of the kernel filter, respectively.

If the frame is not the first frame input after the tracking command, that is, the frame is a second frame or more after the tracking command, the map generator 2500 cross-correlates the feature image with the MOSSE filter, map). That is, the result of the FFT of the five feature images is cross-correlated with the MOSSE filter of the previous frame, respectively. Further, the map generating unit 2500 generates one correlation map from a plurality of correlation maps. Normalization of the five PSRs is performed, and then the normalized PSRs are multiplied to generate one correlation map. That is, the map generator 2500 normalizes the calculated plurality of PSRs once again at a full size, multiplies the normalized correlation maps by the normalized PSRs, and adds the resultant values to generate a correlation map.

The inverse fast Fourier transformer 2600 transforms the frequency-based signal into a time-based signal. The inverse fast Fourier transformer 2600 performs inverse fast Fourier transform (IFFT) on each cross correlation result to normalize the cross correlation between the feature image and the MOSSE filter. That is, IFFT is performed on each of the five cross-correlation results, and then each correlation map is normalized. At this time, the IFFT also applies the IPP library for speedup.

The arithmetic operation unit 2700 detects the peak point of each map and calculates the PSR. That is, the calculation unit 2700 calculates the peak values of the x and y coordinates in the normalized five correlation maps, and then calculates the PSR. In order to calculate the PSR, a peak value in a region excluding 11 11 from the peak point reference for each map is searched and a PSR obtained by measuring the noise level relative to the peak value is calculated. In addition, the operation unit 2700 calculates the final PSR and calculates the subpixel value. That is, the peak value of the x and y coordinates is calculated in one correlation map, the correlation map is normalized, the final PSR is calculated, and the subpixels are calculated based on the x and y values obtained from the correlation map normalization. The final PSR is calculated by finding a peak point with one correlation map finally obtained, and the subpixel value is calculated based on the peak point to derive the final tracking result. In addition, the operation unit 2700 cross-correlates each of the five feature FFT images with the kernel filter FFT image in order to update the MOSSE filter, and weights the cross correlation result obtained from the existing MOSSE filter and the current frame weighted sum) to update the MOSSE filter.

On the other hand, the input unit 3000 converts the image photographed by the photographing device into an electrical signal and provides it. The photographing device may include a lens for photographing, a diaphragm for adjusting the amount of light, and the like, and may include a surveillance camera or the like.

The memory unit 4000 stores data generated in the process of the target detection and tracking method using the target detection and tracking apparatus according to an embodiment of the present invention. For example, the memory unit 400 may store data of the MOSSE filter. At this time, the MOSSE filter data of the memory unit 4000 is updated and stored for each frame of the input image.

The control unit 5000 controls the target detection and tracking apparatus according to an embodiment of the present invention so that the target detection and tracking method is implemented. That is, each of the configurations of the target detection and tracking apparatus described with reference to FIG. 18 is controlled to implement the target detection and tracking method described with reference to FIG. 1 to FIG. The target detection and tracking method has been described with reference to FIGS. 1 to 4, and the subsequent processes are controlled by the control unit 5000, so that the functional description of the control unit 5000 will be omitted.

The target detection and tracking method according to an embodiment of the present invention is tested and the results are described.

Tracking performance was tested for a total of 800 frames using the target detection and tracking method according to the present invention. The target was a drone, about 1 km away, and the test was conducted to track the target on the image passing through the tower. FIG. 21 is a photograph of one frame of 1 to 290 frames, FIG. 22 is a frame picture of 291 to 334 frames, FIG. 23 is a photograph of one frame of 335 to 440 frames, Fig. 24 is a photograph of one frame of 441 to 526 frames, Fig. 25 is a photograph of one frame of 527 to 585 frames, and Fig. 26 is a photograph of one frame of 586 to 800 frames. That is, FIG. 21 is a photograph of the air target, FIG. 22 is a photograph in which the target enters the tower, and FIGS. 23 to 26 are photographs in which the target passes through the tower. 23, 24, and 26 are photographs in which the target is exposed between the spaces of the tower, and FIG. 25 is a photograph in which the target is obscured between the towers.

Experimental results show that, except for pure tracking with no problems, and when the target edge is not clear at the time of entering the tower, that is, when tracking is performed in the memory tracking mode using a linear Kalman filter only for 527 to 585 frames, Tracking was easily performed. That is, as shown in FIG. 25, when the target enters the tower and is obscured by the tower, it is tracked in a memory tracking mode using a linear Kalman filter. In the remaining cases, tracking can be performed in an automatic tracking mode.

Meanwhile, the present invention speeds up the calculation time of detection and tracking using the IPP library for high speed detection and tracking. The environment of the PC that performed the algorithm operation is as follows. FIG. 27 is a graph showing the total execution time of detection and tracking in a total 800 frames of images. The total execution time of the detection and tracking is a sum of the performances of all modules, and time measurement results for each frame are summarized Respectively. The unit is ms (1/1000 second). As shown in FIG. 27, the execution time of the detection is about 2.5 ms to 3.5 ms, and the execution time of the tracking is about 4.6 ms to 6.3 ms. Therefore, it can be confirmed that the computation time is greatly reduced through the high-speed operation using the IPP library.

The present invention is not limited to the above-described embodiments, but may be embodied in various forms. In other words, the above-described embodiments are provided so that the disclosure of the present invention is complete, and those skilled in the art will fully understand the scope of the invention, and the scope of the present invention should be understood by the appended claims . Further, embodiments of the present invention can be combined with one another.

1000: target detection unit 2000: target tracking unit
3000: input unit 4000: memory unit
5000:

Claims (13)

Determining whether a target image is input after a tracking command of the user when the target image is received;
Detecting a target using a plurality of filters if the input image is input without a user's tracking command;
A process in which a user sets a desired target and commands a tracking;
Generating a MOSSE filter for a first frame after a tracking command if the input image is an input image after a user's tracking command;
Calculating a peak to side-lobe ratio (PSR) of a second or more frames input after a user's tracking command and deriving final tracking results;
Comparing the PSR with the threshold value, and if the PSR is greater than the threshold value, updating the MOSSE filter and transiting to the automatic tracking mode; And
And transitioning to the memory tracking mode or the target loss mode according to the target tracking of the set tracking time when the PSR is smaller than the threshold,
The process of generating the MOSSE filter includes:
Acquiring a plurality of feature images;
Generating a kernel filter of a previous frame and performing fast Fourier transform;
Generating a MOSSE filter by cross-correlating a plurality of feature images with a kernel filter,
The step of calculating the PSR includes:
Generating a plurality of correlation maps by cross-correlating a plurality of feature images with a MOSSE filter of a previous frame;
Performing normalization on the plurality of correlation maps by inverse fast Fourier transform,
Calculating a peak value of x, y coordinates in a plurality of normalized correlation maps,
Calculating a plurality of PSRs measuring a noise value relative to a peak value by searching for a peak value excluding a predetermined region on a peak point reference of a plurality of normalized correlation maps,
The step of deriving the final tracking result includes:
Generating a correlation map from a plurality of correlation maps;
Calculating a peak value of x and y coordinates in one correlation map and then calculating a final PSR;
Calculating a subpixel from x and y peak values obtained from one correlation map, and deriving a final tracking result;
Wherein the one correlation map is generated by normalizing a plurality of PSRs for a total size, multiplying the plurality of normalized correlation maps by a normalized PSR, and adding the resultant values to each other.
◈ Claim 2 is abandoned due to payment of registration fee. The method of claim 1, wherein detecting the target comprises:
Acquiring a Gaussian image and a Laplacian image from an input image;
Determining threshold values of the Gaussian image and the Laplacian image, respectively, and binarizing the threshold values,
Combining the Gaussian binarization image and the Laplacian binarization image,
And extracting a boundary of the target from the binarized image to detect a center point.
◈ Claim 3 is abandoned due to the registration fee. The target observation method according to claim 2, wherein the Laplacian image is acquired from the Gaussian image.
◈ Claim 4 is abandoned due to the registration fee. 4. The method of claim 3, further comprising obtaining a histogram of a Gaussian image and obtaining a Laplacian absolute value image before determining the threshold value of the Gaussian image and the Laplacian image.
◈ Claim 5 is abandoned due to the registration fee. The method according to claim 2, wherein the binarization and the combination of the Gaussian image and the Laplacian image are simultaneously performed.
◈ Claim 6 is abandoned due to the registration fee. The method of claim 1, wherein detecting the target comprises:
Acquiring a Gaussian image and obtaining a Laplacian image from a Gaussian image,
Obtaining a histogram of the Gaussian image and obtaining a Laplacian absolute value image;
Determining a threshold value of the Gaussian image and the Laplacian image and binarizing the threshold value,
Combining the Gaussian binarization image and the Laplacian binarization image,
Performing segmentation and labeling to extract a boundary of a target from the binarized image to detect a center point;
Displaying a target candidate and allowing the user to set a desired target.
◈ Claim 7 is abandoned due to registration fee. The method according to claim 6, wherein the process of binarizing the Gaussian image and the Laplacian image and the process of combining the binarized image are simultaneously performed.
◈ Claim 8 is abandoned due to the registration fee. The target observation method according to any one of claims 2 to 7, wherein at least one of the detection processes uses an IPP (Integrated Performance Primitives) library.
delete delete delete ◈ Claim 12 is abandoned due to registration fee. The method of claim 1, wherein updating the MOSSE filter comprises:
Generating a kernel filter at a center of the tracking result derived from the current frame and then performing fast Fourier transform;
A step of cross-correlating a fast Fourier transform result of a plurality of feature images with a fast Fourier transform result of a kernel filter,
And updating the MOSSE filter by weighting the cross correlation result obtained in the current frame with the existing MOSSE filter.
◈ Claim 13 is abandoned due to registration fee. 14. The method of claim 12, wherein the fast Fourier transform and the inverse fast Fourier transform are performed using an IPP (Integrated Performance Primitives) library.

Images