KR101935969B1 - Method and apparatus for detection of failure of object tracking and retracking based on histogram - Google Patents

Abstract

The present invention relates to a method and an apparatus for detecting a failure in tracking an object and retracking using a histogram. The method for sensing a failure in tracking an object using a histogram comprises the steps of: receiving a projection image by an object tracking apparatus; setting an object tracking region based on the projection image by the object tracking apparatus; tracking a tracking object in the object tracking region by the object tracking apparatus; determining object tracking information to track the tracking object by the object tracking apparatus; and detecting a movement of the tracking object on the projection image based on the object tracking information by the object tracking apparatus.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object tracking failure detection and retracking method,

The present invention relates to a method and apparatus for object tracking failure detection and retrace using a histogram. More particularly, the present invention relates to a method and apparatus for quickly detecting whether an object tracking failure occurs through a change in histogram information, and effectively retargeting the object region through re-detection of an object tracking region in case of object tracking failure.

With the development of media technology, the application of media technology to museums and exhibition halls is continuously increasing. Accordingly, the market for beam projectors that output media images is expected to continue to grow.

In the past, the aspect of the exhibition that physically watched the work of a specific artist is now changing to be combined with advanced technology. The 'Quarry of Light' is an example of overseas recreation of previously abandoned space based on media art technique called culture and software mixture.

Lex Baux de Province is a representative example of a successful example of a local government combining cutting-edge science and technology as a venue for art and media art exhibitions. . A huge quarry that has been mined for more than 100 years, named Carrieres de Lumieres, has been abandoned after the abandonment of a limestone cave. It was used as an art exhibition hall.

There are about 70 beam projectors installed on the ceiling inside the exhibition hall, which has a large space unique to the quarry, projecting images on the walls and the whole floor.

In the case of the projection mapping operation, the position of the beam projector changes due to a physical external impact in the field, and the projected surface is frequently distorted. In this case, it leads to the maintenance burden of the construction company and the operating agency. In the case of the hall or the conference room using the beam projector in addition to the projection mapping, the same problem is continuously occurring.

There is an inconvenience that the remote controller operation process must be performed in the case of an expensive beam projector that supports the screen correction function.

Markers can be used to calibrate the beam projector's screen. Accurate tracking of the markers is required to perform marker based corrections. Therefore, it is necessary to study the method for more accurate tracking of the marker.

It is an object of the present invention to solve all the problems described above.

Another object of the present invention is to quickly grasp whether object tracking fails using a histogram.

Another object of the present invention is to quickly detect whether an object tracking failure has occurred using a histogram, and to quickly retrace the object to be tracked.

In order to accomplish the above object, a representative structure of the present invention is as follows.

According to an aspect of the present invention, there is provided a method for detecting an object tracking failure using a histogram, the method comprising: receiving a projection image by an object tracking device; setting an object tracking area based on the projection image; The apparatus comprising: an object tracking unit for tracking a tracking object in the object tracking area; an object tracking unit for determining object tracking information for tracking the tracking object; And detecting movement of the tracking object.

According to another aspect of the present invention, an object tracking apparatus for detecting object tracking failure using a histogram includes an image capturing unit configured to receive a projection image, a tracking object setting unit configured to set an object tracking area based on the projection image, And the tracking device may be configured to track a tracking object in the object tracking area and to detect movement of the tracking object on the projection image based on the object tracking information.

According to the present invention, the object tracking failure can be quickly determined using the histogram.

In addition, the object tracking can be quickly detected using the histogram, and the tracking object can be retraced quickly.

1 is a conceptual diagram illustrating an object tracking failure detection and re-tracking apparatus based on a histogram according to an embodiment of the present invention.
2 is a conceptual diagram illustrating a method of extracting an object feature based on a histogram and tracking an object according to an embodiment of the present invention.
3 is a flowchart illustrating an object tracking failure detection and re-tracking method using a histogram according to an embodiment of the present invention.
4 is a flowchart illustrating an object tracking failure detection and re-tracking method using a histogram according to an embodiment of the present invention.
5 is a flowchart illustrating an object tracking failure detection and re-tracking method using a histogram in an embodiment of the present invention.
6 is a conceptual diagram illustrating the performance of the object tracking failure detection and retrace method using the histogram according to the embodiment of the present invention.
7 is a conceptual diagram illustrating an object re-tracking method according to an embodiment of the present invention.

The following detailed description of the invention refers to the accompanying drawings, which illustrate, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different, but need not be mutually exclusive. For example, the specific shapes, structures, and characteristics described herein may be implemented by changing from one embodiment to another without departing from the spirit and scope of the invention. It should also be understood that the location or arrangement of individual components within each embodiment may be varied without departing from the spirit and scope of the present invention. Therefore, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention should be construed as encompassing the scope of the appended claims and all equivalents thereof. In the drawings, like reference numbers designate the same or similar components throughout the several views.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention.

Conventional image-based object tracking systems have a problem in that when the object to be tracked is hidden or moves quickly, the object to be tracked is missed and the object to be tracked is not tracked again.

In general, the object tracking system uses a slide window (a method of cutting the image according to a specific criterion) with respect to the input image. When considering the number of operations and the computation efficiency, the object tracking system is designed to operate only in a high- have.

In the object tracking failure detection and retrace method and apparatus using the histogram according to the embodiment of the present invention, when a desired object is detected in an image, it is determined whether object tracking failure is based on a change in histogram information of the corresponding region, A method of retargeting a region through re-detection of an area in which object tracking was performed when failure occurs, or a new moving object tracking method according to a histogram change is disclosed.

A histogram according to an exemplary embodiment of the present invention may be a histogram of brightness values of pixels of an image. The histogram may be a graph obtained by setting the intensity value of the image as a horizontal axis and the number of pixels having a size corresponding to the brightness value of the horizontal axis as the number of pixels in the image.

In addition, the object detection according to the embodiment of the present invention may be a method of recognizing and detecting an object existing in a video or image by confirming a region of interest of a pixel classifying the object.

In addition, the object tracking according to the embodiment of the present invention can be used to track the position of an object moving from the image to the next image using the characteristics of the object such as the motion, speed, and color of the object.

The object tracking failure detection and re-tracking method and apparatus using the histogram according to the embodiment of the present invention not only has a system for tracking a marker for correcting a beam projector image, but also tracks a user and controls access only to an authorized user A high-pass system that authenticates via short-range communication, a system that manages violations of road traffic laws, such as speeding and parking, and a monitoring system that monitors personnel using the system.

1 is a conceptual diagram illustrating an object tracking failure detection and re-tracking apparatus based on a histogram according to an embodiment of the present invention.

In FIG. 1, an object tracking failure detection and re-tracking device (hereinafter referred to as an object tracking device) based on a histogram is started.

1, an object tracking apparatus includes an image sensing unit 100, a tracking object setting unit 110, an image processing unit 120, a tracking object determination unit 130, a tracking object retrace setting unit 140, (150).

The image capturing section 100 may be implemented to perform capturing of the tracking object. The image pickup section 100 can perform imaging of the surroundings of the tracked object and the tracked object.

The tracking object setting unit 110 may be performed for setting a tracking object. The tracking object setting unit 110 may be implemented for setting a tracking object to be tracked. The tracking object setting unit 110 may set an area including the tracking object as an object tracking area.

The image processing unit 120 may be implemented to perform image processing on the captured tracking object image. The image processing unit 120 divides the object tracking area, extracts a gradient vector for the divided object tracking area, determines a histogram based on the gradient vector, and performs a fast fourier transform (FFT) on the histogram Object tracking information can be determined. The image processing unit 120 may periodically / non-periodically generate object tracking information and may transmit object tracking information to the tracking object determination unit.

The tracking object determination unit 130 can determine whether tracking of the tracking object is continuously performed in the object tracking area set based on the object tracking information or whether the tracking object is moving on the object tracking area.

The tracking object retarget setting unit 140 may be performed to retrace the tracking object when the tracking object is on the object tracking area as a result of the determination by the tracking object determination unit 130. [ The tracking object retarget setting unit 140 may perform tracking of the tracking object by resetting the object tracking area by determining the position of the tracking object.

The processor 150 may be implemented to control the operations of the image pickup unit 100, the tracking object setting unit 110, the image processing unit 120, the tracking object determination unit 130, .

2 is a conceptual diagram illustrating a method of extracting an object feature based on a histogram and tracking an object according to an embodiment of the present invention.

In FIG. 2, detection of a tracked object is performed, and it is determined whether object tracking is failed based on a change in the histogram value of the tracked object during object tracking, and the object is retraced again .

Referring to FIG. 2, a specific object tracking area 200 is set, and an object tracking area 200 may be divided into a plurality of sub-object tracking areas 220. A gradient vector 240 for a plurality of sub-object tracking areas 220 may be extracted. The extracted gradient vector 240 is a vector composed of the first-order partial derivatives of each variable. And the gradient vector 240 represents the direction in which the value of the gradient of the multivariate function increases most steeply, and the magnitude of the vector may represent a steep degree (slope) of the increase.

The gradient in the image can be used to find the edge and edge direction of the image. Find the size and direction of the gradient at each pixel location to see how close it is to the edge and where the edge is.

The histogram 260 can be extracted based on the gradient vector 240. As described above, the histogram 260 is a histogram of brightness values of pixels of an image, and the brightness value of the image is represented by a horizontal axis, and the number of pixels having a size corresponding to the brightness value of the horizontal axis is indicated in the image It is a graph made by using frequency as a vertical axis.

A fast Fourier transform (FFT) is applied to the histogram 260, and change information of the histogram 260 can be extracted based on the FFT 280. The histogram change information may be object tracking information.

The movement of the tracking object can be determined based on the change of the object tracking information. Specifically, when the tracked object to be imaged is moved in the object tracking area, the histogram 260 determined based on the gradient vector 240 is changed, and the object tracking information can be changed according to the change in the histogram 260. That is, the change of the object tracking information may mean the movement of the tracking object.

Specifically, when there is a change in a value over a weight value on the newly determined histogram compared with the reference histogram 260 determined when the tracking object exists in the object detection area, the histogram 260 is changed to a fatal ) Can be seen.

In this case, when the histogram 260 stops, it is determined that the tracking for the tracking object has failed, and the tracking object detection algorithm can perform the re-detection for the object tracking area.

On the contrary, if a change of the histogram 260 below the threshold value occurs, it is determined that the tracking object is correctly tracked in the object tracking area 200, and tracking can be continuously performed.

In addition, the tracking object detection algorithm utilizes the characteristic that the histogram 260 is changed when a change of the tracking / object detection area of the tracking object is detected (for example, when the frequency of the object tracking information changes) , It is possible to extract and trace the moved coordinates based on the change amount of the object tracking information after the moving object of the tracking object and the moving object of the tracking object.

In the object tracking failure detection and re-tracking method using the histogram 260 according to the embodiment of the present invention, it is possible to efficiently perform object tracking without missing, and it is possible to perform fast tracking of an object to be tracked, It is possible to prevent the tracking failure due to the tracking error, thereby reducing the constraints on the external environment when tracking the object.

In addition, in the object tracking failure detection and retrace method using the histogram according to the embodiment of the present invention, it is possible to track an object even in a limited situation by designing considering the computation efficiency of the embedded environment.

Also, in the object tracking failure detection and retrace method using the histogram according to the embodiment of the present invention, the tracking object can be accurately tracked with fewer operations using the color difference-based algorithm for the image provided by the photographing sensor.

In addition, in the object tracking failure detection and re-tracking method using the histogram according to the embodiment of the present invention, various tracking objects can be tracked as a tracking object tracking system using the minutiae of the color difference of the tracking object.

In addition, moving objects can be tracked from the equipment itself without the aid of remote clients, computing servers, and the like.

3 is a flowchart illustrating an object tracking failure detection and re-tracking method using a histogram according to an embodiment of the present invention.

In FIG. 3, a method of tracking an object by extracting different color difference information with respect to an image frame is disclosed.

Referring to FIG. 3, image frames having different characteristics can be extracted based on different color difference attributes with respect to an input image frame.

A frame based on different color difference information such as LAB frame 310, YCrCb frame 320, and LUV frame 330 can be extracted.

Hereinafter, different image frames extracted based on different image characteristic information (for example, color difference information) for one image frame are divided into a first color difference image frame 310, a second color difference characteristic frame 320, And may be a three-color difference characteristic frame 330. [

Gray scaling 340 is performed for each of the first chrominance characteristic frame 310, the second chrominance characteristic frame 320 and the third chrominance characteristic frame 330 and an AND operation 350 is performed. Can be performed.

After the AND operation 350, detection of the object may be performed based on the dilation 360 and erosion 370 of the image.

Through this procedure, tracking of objects can be performed more accurately through object tracking based on different color difference information.

The AND operation 350 for each of the plurality of chrominance characteristic frames (for example, the first chrominance characteristic frame 310, the second chrominance characteristic frame 320 and the third chrominance characteristic frame 330) Tracking on the object can be performed.

4 is a flowchart illustrating an object tracking failure detection and re-tracking method using a histogram according to an embodiment of the present invention.

FIG. 4 shows a result of performing a method of tracking an object by extracting different color difference information with respect to an image frame.

Referring to FIG. 4, (a) is an original image, (b) is a yCbCr image, (c) is a luv image, (d) is a lab image, and (e) is a mixed image.

More accurate tracking of the tracking object can be performed through the AND operation by tracking the tracking object based on different color differences for one image frame.

5 is a flowchart illustrating an object tracking failure detection and re-tracking method using a histogram in an embodiment of the present invention.

Referring to FIG. 5, image information is input (step S500).

The image can be picked up by the image pickup section. The image capturing for the area including the tracking object is performed for detection of the tracking object, and the image information corresponding thereto can be inputted.

The tracking object may be detected (step S510).

The tracking object can be located within the object tracking area, and detection for the tracking object can be performed. The information about the tracking object can be input and the setting for the object tracking area including the tracking object and the tracking object can be performed based on the information about the tracking object to detect the tracking object.

The coordinates for the object tracking area and / or the tracking object may be set (step S520).

The coordinate information for the object tracking area and the coordinate information for the tracking object can be set on the input image. For example, x-coordinate information and y-coordinate information for the object tracking area and xy coordinate information for the tracking object can be determined by setting x-coordinate and y-coordinate based on the input image.

Object tracking information for the object tracking area and / or the tracking object is extracted (step S530).

The object tracking information is obtained by dividing the object tracking area into a plurality of sub-object detection areas as described above, and then extracting the histogram based on the gradient vector of the plurality of sub-object tracking areas by performing an FFT based on the extracted histogram Information. If there is a change of more than a weight value on a newly determined histogram compared with the reference histogram and there is a change of more than a threshold value, then the histogram is fatal, and based on this phenomenon, Can be detected.

If the amount of change in the object tracking information is larger than the threshold value, the tracking object is re-traced again. If the amount of change in the object tracking information is smaller than or equal to the threshold value, it is determined that the tracking object is normally being tracked. Can be performed.

6 is a conceptual diagram illustrating the performance of the object tracking failure detection and retrace method using the histogram according to the embodiment of the present invention.

FIG. 6 is a conceptual diagram comparing the performance of the conventional object tracking algorithm and the object tracking failure detection and retrace method using the histogram according to the embodiment of the present invention.

Referring to the upper part of FIG. 6, when an obstacle occurs in the middle of the tracking object, tracking for the tracking object fails, and the tracking point sweeps the object.

Referring to the bottom of FIG. 6, even when an obstacle occurs in the middle of the tracked object, the tracked object can be tracked again without tracking the tracked object,

7 is a conceptual diagram illustrating an object re-tracking method according to an embodiment of the present invention.

Figure 7 discloses a method for performing retraining on a tracking object when tracking for the tracking object fails.

Referring to FIG. 7, an object retry area can be set for retrying the tracking object.

The primary object retry area 710 may first be the area set as the original object tracking area. If there is an obstacle in front of the object, the amount of change of the object tracking information may be larger than the threshold even if the object does not move and the object tracking device does not move.

Accordingly, the primary object retention area 710 is set to be the same as the object tracking area, and the object tracking can be performed primarily on the primary object retention area 710. [

The current histogram characteristic (or current object tracking information) extracted from the primary object retrace area 710 is compared with the reference histogram characteristic extracted from the object tracking area (or reference object tracking information) It can be judged whether or not it is possible.

If the current object tracking information and the reference object tracking information are similar to or more than the threshold value, it is determined that the object is re-traced again, and the primary object re-tracking area 710 is set as a new object re- You can track objects based on information.

If re-tracking of the object fails on the primary object retry area 710, the secondary object retry area 720 can be set in consideration of the movable direction of the object and the movable direction of the object tracking device.

The object movable direction may be a direction in which the object can be moved by an external force. For example, if the object is on the desk, it may move left or right rather than down / up. In this case, the left / right direction may be an object movable direction.

The object tracking device movement direction may be a direction in which the object tracking device can be moved by an external force. For example, if the object tracking device is on a desk, it may move left or right rather than down / up. In this case, the left / right direction may be an object movable direction. The direction in which the object can be moved, and the direction in which the object tracking device can be moved can be determined based on the current position of the object and the object tracking device.

The secondary object re-tracking area 720 considering the moving direction of the object and the moving direction of the object tracking device is set to move in the movable direction by the threshold size of the object based on the size of the object, . For example, the secondary object retry area 720 may be set by setting the object moveable direction by a threshold percent (e.g., 100%) of the object size. After setting the secondary object retry area 720, it is determined whether or not the object exists on the secondary object retry area 720. If the object exists, 0.0 > 720 < / RTI >

If the object does not exist on the secondary object retry area 720 after setting the secondary object retry area 720, the object can be moved by twice (for example, 200%) the critical percentage of the object size The secondary object retry area 720 can be reset by setting the direction. Similarly, after the secondary object retry area 720 is reset, it is determined whether or not an object exists on the secondary object retry area 720. If the object exists, the object is re-traced Additional corrections can be made for.

In this manner, if the setting of the n secondary object retry area 720 is not found even after the tracking, the secondary object retry area 720 can be set assuming that the object tracking device has moved next . When the object tracking device moves, a relatively large amount of motion may be generated. In this case, considering the direction in which the object tracking device can move around the primary object retrace area, All regions can be set as the tertiary object retry area 730 to perform retry on the object.

If the retry of an object based on the tertiary object retry area 730 fails, the object tracking device can adjust the zoom magnification of the image and set the object tracking area by searching for the object again at a low zoom magnification.

Also, according to an embodiment of the present invention, a gradient vector, histogram information, and object tracking information may be utilized for object tracking. The determination of an area similar to the existing image based on the gradient vector of the image, the histogram information, and the object tracking information can be performed.

For example, an area having an attribute of a similar gradient vector may be determined based on the attribute of the gradient vector generated by the object. It is possible to perform detection for an area in which similar portions exist by considering the attributes of the gradient vectors of each of the plurality of lower object tracking areas included in the object tracking area. For example, the gradient vector of the 400 sub-object tracking areas exists, and the object tracking information can be extracted. If the object has moved within the object tracking area, there may be a similar gradient vector on the critical area. According to the embodiment of the present invention, the object tracking apparatus compares attributes of a plurality of gradient vectors on a plurality of sub-object tracking regions, extracts a similar gradient vector region having similar properties, and moves the object through the similar gradient vector region Direction. The similar gradient vector region is first analyzed through the default geometric shape, and if a similar gradient vector exists within the critical area of the default geometric shape, the similarity gradient vector region having the highest degree of similarity can be generated by changing the default geometric shape have. A similar gradient vector region having the attributes of the most similar gradient vector may be determined by reducing a portion of the default geometry or extending a portion of the default geometry.

The embodiments of the present invention described above can be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention or may be those known and used by those skilled in the computer software arts. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, medium, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code, such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be modified into one or more software modules for performing the processing according to the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, Those skilled in the art will appreciate that various modifications and changes may be made thereto without departing from the scope of the present invention.

Accordingly, the spirit of the present invention should not be construed as being limited to the above-described embodiments, and all ranges that are equivalent to or equivalent to the claims of the present invention as well as the claims .

Claims (8)

A method for detecting object tracking failure using a histogram,
The object tracking device receiving the projection image;
Setting the object tracking area based on the projection image;
The object tracking device tracking a tracking object within the object tracking area;
The object tracking device determining object tracking information for tracking the tracking object; And
The object tracking apparatus detecting movement of the tracking object on the projection image based on the object tracking information,
Wherein the object tracking information is obtained by dividing the object tracking area into a plurality of subobject detection areas, extracting a histogram based on the gradient vectors for the plurality of subobject tracking areas, and performing fast Fourier transform (FFT) based on the histogram, Lt; RTI ID = 0.0 >
Wherein the detection of movement of the tracking object is performed based on a change in the object tracking information due to a fatal phenomenon of the histogram. delete delete The method according to claim 1,
The object tracking may be performed based on a plurality of color body frames,
Wherein the plurality of color body frames include an LAB frame, a YCrCb frame, and an LUV frame generated based on different color difference information. An object tracking apparatus for detecting an object tracking failure using a histogram,
An image sensing unit configured to receive a projection image;
A tracking object setting unit configured to set an object tracking area based on the projection image;
Wherein the object tracking device is configured to track a tracking object in the object tracking area and to detect movement of the tracking object on the projection image based on the object tracking information,
Wherein the object tracking information is obtained by dividing the object tracking area into a plurality of subobject detection areas, extracting a histogram based on the gradient vectors for the plurality of subobject tracking areas, and performing fast Fourier transform (FFT) based on the histogram, Lt; RTI ID = 0.0 >
Wherein the detection of the movement of the tracking object is performed based on a change in the object tracking information due to a fatal phenomenon of the histogram. delete delete 6. The method of claim 5,
The object tracking may be performed based on a plurality of color body frames,
Wherein the plurality of color body frames include an LAB frame, a YCrCb frame, and an LUV frame generated based on different color difference information.

Images