KR101772928B1 - Transition detector system and method using luminance smoothin and optical flow analysys - Google Patents

Abstract

The present invention relates to a system and a method for movement detection using smoothing and an optical flow analysis, capable of accurate recognition and accurate movement detection of a target object even when an unintended optical noise by a light source or sunlight is generated in a system of detecting movement, movement direction, or speed of the target object. According to the present invention, the system for movement detection using an optical flow analysis comprises: a capturing part including the target object which is the movement detection object, and generating captured image data composed of a plurality of frames; an analyzing part receiving the captured image data from the capturing part, detecting the movement of the target object included in the image by comparison between the frames, determining whether a brightness change greater than or equal to a reference value has occurred on the image, compensating the brightness change to the frames, and comparing the frames; and an input part for setting the reference value.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a motion detection system and method using luminance smoothing and optical flow analysis,

The present invention relates to a movement detection system and method using optical flow analysis, and more particularly to a movement detection system and method using optical flow analysis, in which a light source due to an unintended light source or sunlight is generated in a system for detecting movement, And more particularly, to a motion detection system and method using smoothing and optical flow analysis that enables precise recognition of a target object and accurate detection of movement.

There are various methods for detecting the movement of an object. A method of detecting acceleration of an object, a change of a load, a change of a contact point, a method of using various sensors such as a method of using a change of magnetic force, a method of irradiating a laser or a sound wave to the surface of a target object, , And a method of determining the movement of an object by capturing an image of the object and ascertaining the displacement point of the image.

Among these methods, there is an optical flow method of finding a displacement point by using difference between image frames to photograph an object among methods of using images. In this optical flow method, an image including an object is continuously photographed, and each frame generated by photographing is compared to find a changed part of the image, thereby calculating a motion vector.

In this optical flow method, a motion vector is calculated by reading the change occurrence direction and the change direction of each pixel of the image, and the moving vector, the moving direction, and the speed are detected through the calculation.

In the optical flow method, when a frame is compared, a color change or a brightness change part of a specific pixel is searched. Therefore, a pixel corresponding to a target object in the image of a specific object senses the movement of the pixel on the assumption that the frame does not change .

However, in such an optical flow method, there is a problem that when an image is captured using a camera, the movement of the object can not be detected depending on whether a light source or sunlight is generated. More specifically, when a bright light source appears within a frame range of an image of an object, there occurs a change in luminance due to the light source, and there is a problem that the movement of the object is mistakenly recognized due to a change in the irradiation position of the light in accordance with the movement of the sun .

Korean Patent Laid-Open No. 10-2004-0107962 (published December 23, 2004).

SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a system and method for accurately recognizing and accurately moving a target object even when a light source due to an unintended light source or sunlight is generated in a system for detecting movement, And to provide a motion detection system and method using smoothing and optical flow analysis that enables detection of motion.

According to an aspect of the present invention, there is provided a motion detection system using optical flow analysis, comprising: a photographing unit that generates photographed image data including a subject object as a movement detection target and configured of a plurality of frames; Wherein the control unit receives the photographed image data from the photographing unit, detects movement of the object included in the image by comparing the frames, determines whether a change in brightness over a reference value occurs in the image, An analyzing unit for performing a comparison between the frames by compensating for the reference value, and an input unit for setting the reference value.

The analyzing unit compares a current frame of the plurality of frames with a neighboring frame or an initial frame to calculate a difference value, or an average displacement value obtained by accumulating and accumulating the displacement values, and compares the average displacement value with the reference value. It is determined that a change has occurred.

Wherein the analyzing unit calculates a frame displacement value between the current frame and the immediately preceding frame and calculates an approximate displacement value obtained by approximating the frame displacement value to the average displacement value, And calculating a displacement of the object by compensating the current frame to compare the current frame with another image frame.

Wherein the analyzing unit performs a singular value process for reducing a luminance change according to a brightness change with respect to the displacement value or the average displacement value.

And the analysis unit calculates a comparison or a displacement value between the frames for an observation region that is a part of the image designated through the input unit.

And a plurality of observation regions are set.

According to another aspect of the present invention, there is provided a motion detection method using luminance smoothing and optical flow analysis, the method including: a photographing step of generating photographed image data including a plurality of frames, A compensating step of determining whether a change in brightness of a reference value or more has occurred in the image, and compensating the brightness change for a current one of the plurality of frames; And a comparison step of detecting the movement of the object included in the image by comparing the plurality of frames or the compensated current frame with another frame by receiving the photographed image data from the analysis unit do.

Wherein the compensating step comprises the steps of: calculating a displacement value by comparing the current frame with a neighboring frame or an initial frame; And calculating an average displacement value which is averaged by accumulating the displacement values.

The analyzing unit may further include a step of comparing the displacement value or the average displacement value with the reference value to determine that a change in brightness occurs when the analyzing unit is greater than or equal to the reference value.

Wherein the analyzing unit calculates a frame displacement value between the current frame and the immediately preceding frame and calculates an approximate displacement value obtained by approximating the frame displacement value to the average displacement value, And compensates an approximate displacement value for the current frame, compares the current frame with the other frame, and calculates a displacement of the object.

Wherein the analyzing unit performs the singular value process for reducing the luminance change according to the brightness change with respect to the displacement value or the average displacement value.

Wherein the step of designating an observation region that is a partial region of the image through the input unit is performed, and the compensating step or the comparing step is performed on the observation region.

The motion detection system and method using smoothing and optical flow analysis according to the present invention can be applied to a system for detecting movement such as movement, movement direction, and speed of a target object when a light source due to an unintended light source or sunlight is generated It is possible to accurately recognize and accurately detect the movement of the object.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram illustrating a configuration of a motion detection system using smoothing and optical flow analysis according to the present invention; FIG.
2 is an exemplary diagram for explaining an interpretation of an optical flow;
3 is an exemplary diagram for explaining an influence by a light source;
FIG. 4 is a flowchart briefly illustrating a motion detection method using smoothing and optical flow analysis according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. It should be noted that the drawings denoted by the same reference numerals in the drawings denote the same reference numerals whenever possible, in other drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. And certain features shown in the drawings are to be enlarged or reduced or simplified for ease of explanation, and the drawings and their components are not necessarily drawn to scale. However, those skilled in the art will readily understand these details.

FIG. 1 is a diagram illustrating a configuration of a motion detection system using smoothing and optical flow analysis according to the present invention.

1, a motion detection system using smoothing and optical flow analysis according to the present invention includes a photographing unit 10, an analyzing unit 20 and an input unit 30, The analysis processing unit 20 includes an input unit 11 and an exposure control unit 15. The analysis processing unit 20 includes a frame comparison unit 21, a displacement value generation unit 23, a singular value processing unit 25, an approximate value processing unit 27, And a processing unit (29).

The photographing section 10 photographs the image of the periphery where the object to be detected and the object are installed and transmits the created photographing image data to the analyzing section 20. [ The photographed image data created by the photographing section 10 is composed of a plurality of image frames generated at regular time intervals. In particular, the photographing unit 10 automatically adjusts the luminance of the photographed image according to the luminance value set by the user, thereby creating the photographed image data. To this end, the photographing unit 10 includes an image input unit 11 and an exposure control unit 15.

The image input unit 11 captures an image of a surrounding object in which a target object and a target object are installed, and generates photographed image data through the image.

The exposure control unit 15 controls the exposure of the image input unit 11 and adjusts the brightness of the image so that the brightness of the image data input through the image input unit 11 approaches the value set by the user . The exposure controller 15 adjusts the exposure of the image input unit 11 to adjust the brightness of the photographed image data. For this purpose, the exposure controller 15 receives the brightness value from the input unit 30 and adjusts the exposure amount according to the brightness value . The exposure controller 15 adjusts the brightness or brightness of the image to be equal to the brightness value designated by the user through the gain control and the exposure control for controlling the gain of the image.

The analysis unit 20 receives the photographed image data, performs optical flow analysis to detect the displacement of the image through comparison between frames, and determines movement of the object according to the analyzed displacement. In particular, the analysis unit 20 determines whether or not light noise is generated by the light source on the object and the photographed image data, and analyzes the displacement of the image through a process of filtering the light noise when it occurs. For this, the analysis unit 20 may perform the analysis processing such as Lucas-Canade and Hornque for optical flow processing to detect the displacement of the object. However, the present invention is not limited to this, Any method can be used as long as it is an image analysis method for detecting the movement of the object through the extraction of the displacement through it. Here, the movement includes whether or not the movement has occurred, the movement distance, and the movement speed.

In order to analyze the image by removing the light noise, the analyzing unit 20 determines if the light source is generated in the photographed image data and the brightness of the image is greatly increased, attenuates the brightness increasing effect by the light source, The optical flow is processed using the image data. For this, the analysis unit 20 calculates the displacement value of each image frame and its cumulative average value, determines whether the luminance is increased or not, and analyzes the image using the approximated displacement value when the luminance is rapidly increased . In this process, the analysis unit 20 may process the singular value when calculating the displacement value. Particularly, for this image processing, the analysis unit 20 can perform such optical flow analysis on the region of interest (ROI) set through the input unit 30. The analyzing unit 20 may include a frame comparing unit 21, a displacement value generating unit 23, a singular value processing unit 25, an approximate value processing unit 27, and an area processing unit 29.

The frame comparing unit 21 compares each frame of the photographed image data to detect a displacement generated in the image, and determines the movement of the object in accordance with the displacement detection. To this end, the frame comparing unit 21 compares images of different frames of the image in which the luminance change has not occurred, and compares the compensated frame with another frame in accordance with the luminance increase. In particular, the frame comparing unit 21 compares the image of the observation region set by the user through the input unit 30 to detect the occurrence of the displacement. The frame comparing unit 21 can detect a displacement by reading the difference of each pixel in each frame constituting the image data.

More specifically, the frame comparing unit 21 compares the image frames, which are adjacent to each other or sampled at regular intervals, and compare pixels of each image frame at this time. Here, if the brightness of the image greatly increases and compensation for attenuating the brightness is performed, the frame comparing unit 21 compares the compensated compensation frame with the current frame and performs the optical float analysis. In this case, the compensation frame is generated using the immediately preceding frame of the current frame, and the immediately preceding frame may be a frame adjacent to the current frame continuously or sampled immediately before the current frame. For example, when the frame comparing unit 21 uses the Lucas-Canade method, it is possible to determine whether a displacement occurs through a change in brightness of each pixel. In addition, the occurrence part of a displacement is determined by time persistence and spatial consistency. In particular, the frame comparing unit 21 can perform more accurate analysis with a smaller amount of computation than the conventional method by performing the analysis of the optical flow on a region limited to the observation region. Hereinafter, for ease of explanation, it will be assumed that displacement data is detected by continuously comparing data between neighboring frames among a plurality of frames constituting captured image data.

The displacement value generator 23 generates a displacement map between the neighboring frame or the initial image frame and the current frame. Specifically, the displacement value generator 23 generates a displacement value to determine the brightness change, i.e., the occurrence of the brightness change of the image, and to compensate for the change. Here, the displacement value is calculated by comparing the luminance between neighboring image frames and calculating the difference therebetween. The displacement map means that the displacement value for each pixel is calculated for all the pixels of the image data or all the pixels of the observation region. The displacement value generator 23 calculates a displacement value or a displacement map through comparison between consecutive frames or compares an initial shot image frame with a current frame to calculate a displacement value or a displacement map (hereinafter, the displacement value includes a displacement map (Hereinafter, " used " At this time, the current frame is continuously updated when the shooting continues. On the other hand, the displacement value generator 23 calculates an average displacement value for a plurality of displacement maps when a variation map in which the effect of the light source is reduced by the singular value processor 25 is generated.

The specific value processing unit 25 performs a process of reducing or decreasing a luminance value of a part when the luminance value of a part of the luminance value is greatly changed as in the case where the light source includes the photographed image data. That is, the singular value processor 25 performs a process of reducing a sharp impulse value in a specific portion of the displacement value and a sharp difference in luminance between the other portion and a parameter estimation method such as RANSAC, Robust Parameter Extraction (RPE), m-estimator, But the present invention is not limited thereto. The singular value processor 25 greatly exceeds the brightness of a pixel for displacement estimation and attenuates the change in brightness irrespective of the displacement estimation, thereby reducing the participation of the light source and the resulting light noise.

When the average displacement value is calculated, the approximate value processing unit 27 compares the average displacement value with a predetermined reference value. When the average displacement value is changed to a predetermined level or more, the brightness is changed due to the external condition change, It is determined that the luminance of the pixel is changed. The approximate value processor 27 performs a process of approximating the current frame, which is the image including the light source, and the displacement value generated between the current frame and the immediately preceding frame or between the current frame and the initial frame. That is, the approximate value processor 27 approximates the displacement value of the image frame including the light source to the average displacement value so that the displacement value of the image frame including the light source is represented by a displacement value similar to the image before the light source is included, . For this purpose, the approximate value processor 27 may use an analysis method for generating a displacement value approximated to the average displacement value, such as the least squares method, but the present invention is not limited thereto.

The region processing unit 29 sets the observation region of the photographed image data according to the region designation command input through the input unit 30 and transmits the observation region to the frame comparing unit 21 or the displacement value generating unit 23 . The observation region set in the region processing unit 29 may be composed of a plurality of observation regions, but the present invention is not limited thereto.

The input unit 30 receives a command from the user and transmits the command to the photographing unit 10 or the analysis unit 20. The input unit 30 receives the brightness reference value from the user and transmits the brightness reference value to the exposure control unit 15 of the photographing unit 10 and receives a reference value for determining whether the light source is generated, ). The input unit 30 receives the observation area setting command and transmits the observation area setting command to the area processing unit 29 of the analysis unit 20.

FIGS. 2 and 3 are illustrations for explaining the influence of the optical flow and the light source. FIG. 2 is an exemplary diagram for explaining the analysis of the optical flow, and FIG. 3 is an example for explaining the influence of the light source .

Referring to FIG. 2, the optical flow analysis of the present invention detects the displacement occurring in the photographed image, and detects the movement of the object in which the displacement occurs. At this time, it is judged through detection of displacement such as whether the object is moved, the moving speed, the distance of the object, and the moving time.

To this end, the present invention compares the current image frame generated by image capturing with the first frame or the previous frame to determine whether a displacement occurs. For example, as shown in FIG. 2, there may be a case where the object 52 is present in the photographed image 51, and the movement 53 of the object 52 occurs due to the displacement. In this case, when the displacement is calculated with respect to the movement of the object 52, it becomes possible to indicate the direction and the distance with the region where the displacement occurs as a vector with respect to the portion where the displacement occurs as shown in (b) May be displayed in a state where the same state as before is maintained.

This determination of the occurrence of displacement can be made by tracking pixels with a change in brightness when using the Lucas-Canade method. For example, when using the Lucas-Kanade method, a pixel on a certain object does not change its value even if the frame is changed. That is, the brightness constant of the object pixel is constant, and the amount of change is small compared to the moving time in the image Is analyzed by the concept of spatial coherence that temporal persistence or small movements and spatially adjacent points are likely to belong to the same object and have the same motion. That is, if a pixel in which a brightness or a color change is generated is tracked in successive frames when a comparison is made on a pixel basis constituting a photographed image or on a group basis in which a few pixels are grouped, displaced pixels can be found in the image, It is possible to detect the movement of the object when they are in a spatially related relationship.

However, since brightness uniformity is assumed as described above, if brightness changes occur due to a light source or the like, it becomes difficult to detect the displacement of such a moving object.

3, when a light source 54 such as a sun or an artificial light is generated in the photographing space as shown in FIG. 2, the luminance of the image is generally increased, and the image input unit 11 of the photographing unit 10, Adjusts the exposure so that the luminance of the image approaches the luminance reference value determined by the user. In this case, the process of matching the average brightness of the entire screen with the luminance reference value by the exposure control causes the image of the portion excluding the light source to become dark, and when the changed image is used for the optical flow analysis, There is a problem.

In this case, as in (b), the displacement point by the light source is calculated instead of the object, and the tracking of the object fails.

Furthermore, when the altitude and the irradiation direction continuously change with time as in the sun, the error of the displacement point analysis due to the change of the irradiation direction and the intensity of the light rather than the movement of the object frequently occurs.

Therefore, in the present invention, when the light source is generated, a process of eliminating the effect of the light source is performed, and by calculating the displacement point, the movement of the object is continuously and stably performed.

More specifically, as described above, a displacement value is calculated for each frame and averaged to determine whether or not a brightness change of the photographed image has occurred. When brightness change occurs, The image frame reduced in the frame is compared with the next image frame. Particularly, in the case of such an approximated frame, even when the brightness is simply brighter, the luminance change effect by the light source can still be included in the image data. Therefore, the average value of the displacement values, that is, By approximating the luminance distribution so as to have a luminance distribution similar to that of the luminance distribution, the displacement point by the light source can be minimized and the displacement of the actual object can be detected irrespective of the light source, and the movement detection of the object can be stably performed.

FIG. 4 is a flowchart briefly illustrating a motion detection method using smoothing and optical flow analysis according to the present invention.

Referring to FIG. 4, a motion detection method using smoothing and optical flow analysis according to the present invention includes an image acquisition step S10, a displacement map calculation step S20, a singular value processing step S30, S40, an approximate value calculating step S50, a current frame applying step S60, and a frame comparing step S70.

The image acquisition step S10 is a step in which the image input unit 11 of the photographing unit 10 acquires an image of an object and transmits the acquired image data to the analysis unit 20. [ In the image acquisition step S10, the image input unit 11 may automatically adjust the exposure or gain of the image according to the brightness reference value set by the user so that the average brightness of the image is adjusted to a value close to the brightness reference value.

 The displacement map calculation step S20 is a step in which the displacement value generation unit 23 of the analysis unit 20 calculates a displacement value between neighboring frames or between the initial frame and the current frame in the photographed image data. In the displacement map calculation step S20, the displacement value generation unit 23 calculates the luminance change of the previous frame and the current frame for each pixel to generate the displacement value. Particularly, when the observation region is designated when the displacement value is generated, the displacement value is calculated only in the observation region. Also, the displacement map calculation step S20 may calculate an average displacement value obtained by accumulating displacement values calculated between frames.

In the singular value processing step S30, a process of reducing or decreasing the brightness variation occurs when a light source changes a displacement value between frames to a large extent. This singular value processing step S30 attenuates the change in brightness irrespective of the displacement estimation, thereby reducing the light noise caused by the light source. The singular value processing step S30 may be performed on the inter-frame displacement value after the calculation of the inter-frame displacement value and may be performed on the average displacement value after the calculation of the average displacement value. However, in the actual data processing, It is possible for the user to perform arbitrary placement at a possible stage, and the present invention is not limited to the present invention only.

The comparison step S40 according to the reference value compares the calculated average displacement value with a preset reference value to determine whether light noise has occurred due to the light source. If it is determined that no light noise is generated in the comparison step S40 based on the reference value, the displacement map calculation step S20, the singular value processing step S30 and the frame comparison step S70 are performed, The approximate value calculation step S50 is performed.

The approximate value calculation step S50 approximates the inter-frame displacement value generated for the current frame by using the average displacement value calculated in the displacement value calculation step S20.

 In the current frame application step S60, the approximated displacement value is added to the current frame to remove the light noise generated in the current frame. That is, it is determined that a light noise, that is, a brightness change, has occurred in the current frame in the above-described step, and the brightness change value due to the light noise is determined through the singular value processing step S30 and the approximate value calculating step S50 The compensation value is calculated and applied to the current frame.

 In the frame comparing step S70, the movement of the target object is sensed by comparing the neighboring frame or the first frame with the current frame. In particular, when light noise is generated by the light source, the frame comparing step S70 compares the current frame to which the compensation is applied, the neighboring frame or the first frame to be compared, and compares the frames excluding the light noise by the light source, The movement of the object is detected.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, . ≪ / RTI > Accordingly, such modifications are deemed to be within the scope of the present invention, and the scope of the present invention should be determined by the following claims.

10: photographing section 11: image input section
15: Exposure control section 20: Analysis section
21: frame comparing unit 23: displacement value generating unit
25: singular value processor 27: approximate value processor
29: area processing unit 30:
51: photographing screen 52: target object
54: Light source

Claims (12)

An imaging unit that generates photographed image data including a plurality of frames including an object to be detected as a subject of movement detection;
The moving image sensing unit receives the photographed image data from the photographing unit, detects movement of the object by comparing pixel differences corresponding to minutiae points for recognizing the object, and determines whether or not a change in brightness over a reference value occurs An analyzer for compensating for the brightness change to the frame to perform a comparison between the frames,
And an input unit for setting the reference value,
The analyzing unit compares a current frame of the plurality of frames with a neighboring frame or an initial frame to calculate a difference value, or an average displacement value obtained by accumulating and accumulating the displacement values, and compares the average displacement value with the reference value. It is determined that a change has occurred,
Wherein the analyzing unit calculates a frame displacement value between the current frame and the immediately preceding frame and calculates an approximate displacement value obtained by approximating the frame displacement value to the average displacement value, Wherein the displacement of the object is calculated by compensating the current frame and comparing the current frame with another image frame to calculate a displacement of the object. delete delete The method according to claim 1,
The analyzer
And performing a singular value process for reducing the luminance change according to the brightness change with respect to the displacement value or the average displacement value. The method according to claim 1,
Wherein the analyzing unit compares the frames or calculates a displacement value with respect to an observation area that is a part of the image designated through the input unit, using the luminance smoothing and optical flow analysis. 6. The method of claim 5,
Wherein the observation region is set to a plurality of observation regions. An imaging step of generating photographed image data including a plurality of frames including an object to be photographed, which is an object of movement detection;
A compensating step of determining whether a change in brightness of a reference value or more has occurred in the image, and compensating the brightness change for a current one of the plurality of frames; And
And a comparison step of comparing the difference between the pixels corresponding to the feature points for recognizing the object and receiving the captured image data to detect the movement of the object,
Wherein the compensating step comprises the steps of: calculating a displacement value by comparing the current frame with a neighboring frame or an initial frame; And calculating an average displacement value that is averaged by accumulating the displacement values,
Wherein the compensating step comprises the step of comparing the displacement value or the average displacement value with the reference value and determining that a brightness change has occurred when the analyzing unit is equal to or greater than the reference value,
Wherein the analyzing unit calculates a frame displacement value between the current frame and the immediately preceding frame and calculates an approximate displacement value obtained by approximating the frame displacement value to the average displacement value, Calculating a displacement of the object by compensating an approximate displacement value for the current frame and comparing the current frame and another frame; and calculating a displacement of the object by using the smoothing and optical flow analysis. delete delete delete 8. The method of claim 7,
The compensation step
Wherein the analysis unit performs a singular value process for reducing a luminance change according to a change in brightness with respect to the displacement value or the average displacement value. 12. The method of claim 11,
Further comprising the step of designating an observation region that is a part of the image through an input unit,
Wherein the compensating step or the comparing step is performed for the observation area.

Images