KR20110092781A - Image processing method and apparatus for a motion tracking - Google Patents

Abstract

PURPOSE: An image processing method and apparatus for motion tracking are provided to increase the reliability of motion data due to using previous data and to skip tracking within a frame period. CONSTITUTION: A shaking compensation unit(110) detects the shake of an input image. The shaking compensation unit compensates the shake of the input image in case shake is detected. A tracking unit(130) extracts motion data from the input image. The tracking unit performs motion tracking based on the extracted motion data.

Description

Image processing method and apparatus for a motion tracking}

The present invention relates to an image processing method and apparatus, and more particularly, to an image processing method and apparatus for motion tracking.

Motion Tracking is an upgraded concept from the existing motion detection. It gives a label to the motion on the screen and keeps the same label until it disappears from the screen area. It is used for management, measuring the amount of movement of vehicles and people.

According to an embodiment of the present invention, the motion portion generated due to the camera shake is compensated primarily by the shake compensator to use only the motion data due to the appearance of the object as the tracking data except the shake caused by the shake, and the shake compensator Incoming motion data that is not compensated by the limit is skipped within a frame period in which the tracking algorithm itself can be tracked even without motion, and the previous data is used as it is, thereby providing an image processing method and apparatus for motion tracking that can increase the reliability of motion data. To provide.

According to an aspect of the present invention, there is provided a method of processing an image for motion tracking, the method comprising: detecting shaking of an input image; compensating for shaking of the input image when detecting the shaking; And extracting motion data from the compensated input image and performing motion tracking based on the extracted motion data.

In the performing of the motion tracking, the motion data is extracted from the compensated input image according to the reliability of the compensation of the shake in the compensating step.

The compensation area of the shake in the compensation step may be a pixel area for extracting motion data from the input image.

The image processing method may include increasing a count of a frame of the input image, comparing the count value with a second threshold value when the degree of compensation of the shake is greater than or equal to a first threshold value, and wherein the count value is smaller than the second threshold value. In this case, the method may further include performing motion tracking based on the motion data extracted from the previous frame of the frame.

If the count value is larger than the second threshold, motion tracking is stopped.

The second threshold may be the number of frames capable of motion tracking even without continuous extraction of motion data from the input image.

In order to achieve the above technical problem, an image processing apparatus for motion tracking according to another embodiment of the present invention detects a shake of an input image and, when detecting the shake, shake compensation to compensate for the shake of the input image. And a tracking unit extracting motion data from the compensated input image and performing motion tracking based on the extracted motion data.

The image processing apparatus may further include a reliability determiner that determines a reliability of compensation of the shake in the shake compensation unit, and the tracking unit extracts motion data from the compensated input image according to the reliability of the shake compensation. .

The tracking unit extracts motion data from the shake-compensated input image output from the shake compensation unit, a grouping unit for labeling the extracted motion data by connecting to adjacent pixels, and consecutively grouping each of the labeled groups. And an output unit for tracking and outputting on a frame of an image.

When the degree of compensation of the shake is greater than or equal to a first threshold, the reliability determining unit increases the count of the frame of the input image, compares the count value with a second threshold value, and the tracking unit compares the count value with the second threshold value. In a small case, motion tracking may be performed based on motion data extracted from a previous frame of the frame.

The tracking unit may stop motion tracking when the count value is greater than the second threshold value.

The shake compensation unit performs shake compensation in a pixel area for extracting motion data from the input image.

The second threshold may be the number of frames capable of motion tracking even without continuous extraction of motion data from the input image.

In the image processing method for motion tracking according to an embodiment of the present invention, the motion portion generated due to the camera shake is primarily compensated by the shake compensator so that only the motion data due to the appearance of the object is excluded except the motion caused by the shake. Can be used as tracking data.

In addition, the motion data that is not compensated by the limitation of the shake compensation unit may be skipped within the frame period in which the tracking algorithm itself does not have motion and can use the previous data as it is, thereby increasing the reliability of the motion data.

1 is a schematic block diagram of an image processing apparatus 100 for motion tracking according to an exemplary embodiment.
2A to 2C are exemplary diagrams for describing performing motion tracking through the image processing apparatus 100 illustrated in FIG. 1.
3 is a flowchart illustrating an image processing method for motion tracking according to another embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, only parts necessary for understanding the operation according to the present invention will be described, and descriptions of other parts may be omitted so as not to distract from the gist of the present invention.

In addition, terms and words used in the following description and claims should not be construed to be limited to ordinary or dictionary meanings, but are to be construed in a manner consistent with the technical idea of the present invention As well as the concept.

As used throughout the specification of the present invention, motion tracking refers to an image tracking technique that gives a label to a motion generated on a screen displaying an input image and maintains the same label until it disappears from the screen region. do.

1 is a schematic block diagram of an image processing apparatus 100 for motion tracking according to an exemplary embodiment.

Referring to FIG. 1, an image processing apparatus 100 according to an exemplary embodiment of the present disclosure includes a shake compensation unit 110, a reliability determination unit 120, and a tracking unit 130, and the tracking unit 130 The motion data extractor 131, the grouping unit 132, and the output unit 133 are included. The image processing apparatus 100 according to an exemplary embodiment may be a digital signal processor or an image signal processor included in an image surveillance system (not shown). The image processing apparatus 100 may further include an imaging device, an AD converter, a DA converter, a memory, and the like.

The shake compensator 110 detects a shake of the input image and, when detecting a shake, compensates for the shake of the input image. Here, the input image is an image in which an image signal converted into an electrical signal through an imaging device, for example, a CCD or CMOS image sensor, is sampled as a digital signal. Here, the shaking is caused by the shaking of the camera taking the image. For example, in the case of the surveillance camera installed in a high place where the field of view is secured for traffic relations, an image having a lot of shaking due to the influence of wind is input. In one embodiment of the present invention, the shake compensation unit 110 serves to remove the influence of the shake of the camera on the input image first. Therefore, the image tracking may be performed by extracting only the movement of the object itself to be tracked as motion data in a state in which the influence of the external element, that is, the camera shake, is not the movement of the specific object itself.

Image stabilization can be performed through digital image processing. For example, the motion can be compensated for by shaking the current frame in a direction opposite to the motion vector by obtaining a motion vector expressing the amount of movement that differs from the previous frame and the current frame in the direction and magnitude. As shown in FIG. 2A, the current input image is moved in the direction of the arrow due to the shake of the camera. As shown in FIG. 2B, the shake compensation unit 110 calculates the direction and the size of the arrow in the opposite direction. By moving the image to eliminate the effects of shaking. As shown in FIG. 2C, after performing the shake compensation, only the object of interest is detected to track the movement of the object 220. In an embodiment of the present invention, the motion vector is calculated to compensate for the shake, but the difference between the current image and the previous image is detected by performing other image processing, for example, correlation matching, pattern matching, and color matching. You can also correct this.

In addition, the shake compensation unit 110 may perform the compensation in the pixels of the entire input image, but in a constant compensation pixel area, for example, in a pixel area for finding a motion vector, that is, in a 32 × 32 pixel area. In the case of crossing the pixel area for detecting the motion vector, the shake compensation may be performed within a maximum value, for example, a 128 × 128 pixel area. Therefore, appropriate shake compensation can be performed without causing a problem that the screen changes drastically due to excessive compensation.

The reliability determination unit 120 determines the reliability of compensation of the shake in the shake compensation unit 110. The reliability determiner 120 determines the shifted information of the image according to the limitation of the shake compensation so that the image that is not shake compensated is not used for motion data extraction for tracking. That is, the current frame image is excluded from the motion data extraction when the amount of shake compensation is greater than the threshold, for example, by comparing the information about how many pixels to shift the image of the current frame and a specific threshold.

The tracking unit 130 extracts motion data from the shake compensated input image output from the shake compensation unit 110 and performs motion tracking based on the extracted motion data. Here, the tracking process is a motion data extraction process for extracting a motion using a luminance difference between a current frame and a previous frame, and the extracted motion data is pre-processed to remove noise that removes a small area of a pixel unit determined as noise. Labeling process to extract and group the extracted motion data with adjacent pixels. Each labeled group has a motion vector on frames of consecutive images, and the group is first grouped using the motion vector. The label value includes a process of continuously outputting the group until the group disappears from the screen.

The motion data extractor 131 extracts motion data from the shake compensated input image output from the shake compensator. In an embodiment of the present invention, the motion data extractor 131 extracts the motion data from the shake-compensated input image without extracting the motion data from all the input images. This can compensate for the difference in data reliability between frames.

The grouping unit 132 labels the extracted motion data with adjacent pixels. According to an embodiment of the present invention, if the compensation is impossible due to the limitation of the shake compensation or the video is not completely compensated, a certain number of frames are counted and motion data is not extracted from the current input frame when the frame is within a threshold frame. Labeling is performed using the motion data of the previous frame. Here, the threshold frame is the number of frames that can be tracked without motion data in the tracking algorithm itself.

The output unit 133 tracks and outputs each labeled group on a frame of a continuous image.

In addition, when the degree of compensation of the shake is greater than or equal to the first threshold, the reliability determiner 120 increases the count of the frame of the input image and compares the count value with the second threshold. Here, the second threshold means the number of frames that can be tracked without extracting the motion data of the current frame. When the reliability determiner 120 determines that the count value is smaller than the second threshold value, the tracking unit 120 performs motion tracking based on the motion data extracted from the previous frame of the current frame. On the other hand, if the count value is greater than the second threshold, motion tracking stops. An image processing method related to this will be described in detail with reference to FIG. 3.

3 is a flowchart illustrating an image processing method for motion tracking according to another embodiment of the present invention.

Referring to FIG. 3, first, in operation 300, an image is input through a camera unit. In step 302, if there is no camera shake or camera shake, the process proceeds to steps 308 and 310 to extract motion data and perform tracking. In step 302, if there is a shake of the camera, in step 304 the shake of the camera is compensated, and in step 306, it is determined whether full compensation has been made. That is, it is determined how much the current frame is shifted through the compensation process. In the case of complete compensation, the process proceeds to steps 308 and 310 to extract the motion data and perform tracking. If not completely compensated, in step 314, the frame count of the input image is increased by one. In step 316, it is determined whether the accumulated count value is greater than the threshold value. If the accumulated count value is greater than the threshold, tracking is stopped at step 320 and 322, refreshing the labeling and ending. Here, the threshold is the number of frames that can be tracked without the motion data extracted from the current frame, as described above.

In step 316, if the cumulative count value is smaller than the threshold value, the flow proceeds to step 318 where the motion data extracted from the previous frame instead of the current frame is used as the tracking data. In operation 310, tracking is performed using motion data of a previous frame. In step 312, if tracking is not complete, the flow returns to step 300 again.

In the image processing method for motion tracking according to an embodiment of the present invention, the motion portion generated due to the camera shake is primarily compensated by the shake compensator so that only the motion data due to the appearance of the object is excluded except the motion caused by the shake. Used as tracking data. In addition, the motion data that is not compensated by the limitation of the shake compensation unit may be skipped within the frame period in which the tracking algorithm itself does not have motion and can use the previous data as it is, thereby increasing the reliability of the motion data.

An apparatus according to the present invention may include a processor, a memory for storing and executing program data, a permanent storage such as a disk drive, a communication port for communicating with an external device, a user interface such as a touch panel, a key, Devices, and the like. Methods implemented by software modules or algorithms may be stored on a computer readable recording medium as computer readable codes or program instructions executable on the processor. Here, the computer-readable recording medium may be a magnetic storage medium such as a read-only memory (ROM), a random-access memory (RAM), a floppy disk, a hard disk, ), And a DVD (Digital Versatile Disc). The computer-readable recording medium may be distributed over networked computer systems so that computer readable code can be stored and executed in a distributed manner. The medium is readable by a computer, stored in a memory, and executable on a processor.

All documents including publications, patent applications, patents, etc. cited in the present invention can be incorporated into the present invention in the same manner as each cited document individually and concretely, .

In order to facilitate understanding of the present invention, reference will be made to the preferred embodiments shown in the drawings, and specific terminology is used to describe the embodiments of the present invention. However, the present invention is not limited to the specific terminology, Lt; / RTI > may include all elements commonly conceivable by those skilled in the art.

The invention can be represented by functional block configurations and various processing steps. Such functional blocks may be implemented in various numbers of hardware or / and software configurations that perform particular functions. For example, the present invention relates to integrated circuit configurations such as memory, processing, logic, look-up table, etc., which may execute various functions by the control of one or more microprocessors or other control devices. It can be adopted. Similar to the components of the present invention that may be implemented with software programming or software components, the present invention may be implemented as a combination of C, C ++, and C ++, including various algorithms implemented with data structures, processes, routines, , Java (Java), assembler, and the like. The functional aspects may be implemented with an algorithm running on one or more processors. In addition, the present invention may employ the prior art for electronic environment setting, signal processing, and / or data processing. Terms such as "mechanism", "element", "means" and "configuration" can be used widely and are not limited to mechanical and physical configurations. The term may include the meaning of a series of routines of software in conjunction with a processor or the like.

The specific acts described in the present invention are, by way of example, not intended to limit the scope of the invention in any way. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. Also, the connections or connecting members of the lines between the components shown in the figures are illustrative of functional connections and / or physical or circuit connections, which may be replaced or additionally provided by a variety of functional connections, physical Connection, or circuit connections. Also, unless explicitly mentioned, such as " essential ", " importantly ", etc., it may not be a necessary component for application of the present invention.

The use of the terms " above " and similar indication words in the specification of the present invention (particularly in the claims) may refer to both singular and plural. In addition, in the present invention, when a range is described, it includes the invention to which the individual values belonging to the above range are applied (unless there is contradiction thereto), and each individual value constituting the above range is described in the detailed description of the invention The same. Finally, the steps may be performed in any suitable order, unless explicitly stated or contrary to the description of the steps constituting the method according to the invention. The present invention is not necessarily limited to the order of description of the above steps. The use of all examples or exemplary language (e.g., etc.) in this invention is for the purpose of describing the invention in detail and is not to be construed as a limitation on the scope of the invention, It is not. In addition, one of ordinary skill in the art appreciates that various modifications, combinations and changes can be made depending on design conditions and factors within the scope of the appended claims or equivalents thereof.

Claims (14)

Detecting shaking of the input image;
Compensating for the shaking of the input image when the shaking is detected;
Extracting motion data from the compensated input image; And
And performing motion tracking based on the extracted motion data. The method of claim 1,
And extracting motion data from the compensated input image according to the reliability of the compensation of the shake in the compensating step. The method of claim 1,
The compensation region of the shake in the compensation step is a pixel region for extracting the motion data from the input image. The method of claim 2,
When the degree of compensation of the shake is greater than or equal to the first threshold,
Incrementing a count of frames of the input image and comparing the count value with a second threshold value; And
And if the count value is smaller than the second threshold, performing motion tracking based on motion data extracted from a previous frame of the frame. The method of claim 4, wherein
And stopping the motion tracking when the count value is greater than the second threshold value. The method of claim 5, wherein
The second threshold is,
An image processing method for motion tracking, wherein the number of frames for which motion tracking is possible even without continuous motion data extraction from an input image.  A recording medium having recorded thereon a program for executing a method according to any one of claims 1 to 6 on a computer. A shake compensator for detecting a shake of an input image and compensating for shake of the input image when the shake is detected; And
And a tracking unit extracting motion data from the compensated input image and performing motion tracking based on the extracted motion data. The method of claim 8,
Further comprising a reliability determination unit for determining the reliability of compensation of the shake in the shake compensation unit,
The tracking unit,
And extracting motion data from the compensated input image according to the reliability of the shake compensation. The method of claim 8,
The tracking unit,
A motion data extracting unit extracting motion data from the shake compensated input image output from the shake compensating unit;
A grouping unit to label the extracted motion data with an adjacent pixel; And
And an output unit for tracking and outputting each of the labeled groups on a frame of a continuous image. The method of claim 9,
The reliability determination unit,
When the degree of compensation of the shake is greater than or equal to a first threshold, the count of the frame of the input image is increased, and the count value is compared with a second threshold value.
The tracking unit,
And when the count value is smaller than the second threshold, motion tracking based on motion data extracted from a previous frame of the frame. The method of claim 11,
The tracking unit,
And stop the motion tracking when the count value is greater than the second threshold value. The method of claim 8,
The shake compensation unit,
And image stabilization in the pixel area for extracting motion data from the input image. The method of claim 12,
The second threshold is,
An image processing apparatus for motion tracking, wherein the number of frames for which motion tracking is possible even without continuous motion data extraction from an input image.

Images