KR101214000B1 - Apparatus for detecting color rolling and method thereof - Google Patents

Abstract

PURPOSE: An apparatus for detecting color rolling and a method thereof are provided to detect a color rolling effect according to presumed light source information by estimating light source information based on global image change due to movement of a camera and local image change due to movement of an object. CONSTITUTION: A motion estimator estimates global motion among image frames of an input sequence. A background region extracting unit extracts a common background region without movement among image frames using a presumed global motion. A light source information estimation unit estimates set light source information by using an extracted common background region. A rolling effect detection unit detects a color rolling effect by using change of the light source information. [Reference numerals] (110) Image input unit; (120) Motion estimator; (130) Background region extracting unit; (140) Light source information estimation unit; (150) Rolling effect detection unit; (160) Cycle estimator

Description

Apparatus and method for detecting color rolling {APPARATUS FOR DETECTING COLOR ROLLING AND METHOD THEREOF}

The present invention relates to a color rolling detection method, and more particularly, to estimate light source information in consideration of global image change due to camera movement and local image change due to object movement, and to detect color rolling phenomenon according to the estimated light source information. The present invention relates to an apparatus and a method for detecting color rolling which enable to estimate the color rolling period and amplitude after detecting the color rolling phenomenon.

Since the human visual system is sensitive to changes in the time axis, it is important to secure stability based on the similarity between images of adjacent frames when acquiring a good quality video. When the video is taken indoors, there is a problem that the amount of light accumulated changes when there is a difference between the period of the light source periodically emitting light and the sampling period of the camera system. This is a factor that changes the color in the moving picture sequence, the phenomenon in which the color changes periodically in the moving picture is called a color rolling phenomenon.

For example, if a picture is taken using a camera system with a sampling frequency of 59.94Hz from a fluorescent light source powered by a frequency of 60 Hz, the amount of light acquired is different for each frame, and the overall color of the image is yellow. The color changes periodically to blue and achromatic.

In this case, due to the impairment of the homeostasis of the video, it is not only able to provide high-quality video to people who are visually exposed to the video, but also causes unexpected errors in the process of extracting the region of interest using the video in a security system. do. Therefore, it is necessary to accurately detect such a phenomenon from the input video so as to cope with an error occurring later.

As a method for detecting a color rolling phenomenon, it may depend on an external device. In other words, by using an external sensor to detect the change in the amount of light and send it to the camera system. In this case, although the accuracy of color rolling detection is shown, there is a disadvantage that the cost of the product increases and the volume of the camera increases due to the use of a sensor.

A method for analyzing the input image without using an external device and detecting the signal is also presented. This method does not accurately reflect the color rolling phenomenon by estimating using only the B channel change, which is the largest change in the color rolling phenomenon. Rather, when the blue object is included, a problem of inaccuracy occurs.

Therefore, to solve the problems of the prior art, an object of the present invention is to estimate the light source information in consideration of the global image change due to the movement of the camera and the local image change due to the movement of the object and color according to the estimated light source information The present invention provides an apparatus and method for detecting color rolling that can detect a rolling phenomenon.

In addition, another object of the present invention is to estimate the light source information in consideration of the global image change due to the movement of the camera and the local image change due to the movement of the object and the color rolling period after detecting the color rolling phenomenon according to the estimated light source information An apparatus and method are provided for detecting color rolling that enable to estimate the amplitude and amplitude.

However, the objects of the present invention are not limited to those mentioned above, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

In order to achieve the above objects, an apparatus for detecting color rolling according to an aspect of the present invention includes a motion estimation unit for estimating the global motion between a plurality of image frames of the input image sequence; A background region extraction unit configured to extract a common background region without movement between a plurality of image frames by using the estimated global movement; A light source information estimator configured to estimate preset light source information using the extracted background region; And a rolling phenomenon detection unit configured to detect a color rolling phenomenon by using a change amount of the light source information between the plurality of image frames.

Preferably, the motion estimator extracts global motion information for a predetermined number of previous image frames from the current image frame, and estimates a common area between a plurality of image frames by using the extracted global motion information. .

Preferably, the background region extracting unit obtains a background region for each R, G, and B channel with respect to a common region among the plurality of image frames, and uses the background region for each R, G, and B channel to apply to each image frame. The background area of the image frame may be obtained, and a common background area of previous image frames may be extracted based on the current image frame using the background area of each image frame.

Preferably, the light source information estimating unit obtains a brightness component for each of the R, G, and B channels using the extracted background region, and uses the obtained brightness component for each R, G, and B channel, The brightness component can be estimated.

Preferably, the light source information estimator may obtain a brightness component for each of the R, G, and B channels by averaging the accumulation of edge gradients of each image frame with respect to a common background area of the plurality of image frames.

Preferably, the light source information may include at least one of a brightness component and a color component of the light source appearing in each of the plurality of image frames.

Preferably, the rolling phenomenon detection unit may detect the color rolling phenomenon by using the brightness change amount and the color change amount on the time axis which are the estimated light source information.

The apparatus for detecting color rolling according to the present invention may further include a period estimator for continuously detecting the detected color rolling phenomenon and detecting a period and an amplitude of the color rolling phenomenon.

Preferably, the period estimator detects a time point at which the accumulation of brightness differences of adjacent image frames of each R, G, and B channel is '0' at the same time, and the maximum value of the brightness component obtained within one detected period. The minimum value can be used to detect the amplitude.

An apparatus for detecting color rolling according to another aspect of the present invention includes a background region extraction unit for extracting a common background region without movement between a plurality of image frames of the input image sequence; A light source information estimator configured to estimate preset light source information using the extracted background region; And a rolling phenomenon detection unit configured to detect a color rolling phenomenon by using a change amount of the light source information between the plurality of image frames.

According to another aspect of the present invention, a method for detecting color rolling includes estimating global motion between a plurality of image frames of an input image sequence; Extracting a common background region having no motion between a plurality of image frames by using the estimated global motion; Estimating preset light source information by using the extracted background region; And detecting a color rolling phenomenon by using a change amount of the light source information between the plurality of image frames.

Preferably, the estimating the global motion extracts global motion information for a predetermined number of previous image frames adjacent from a current image frame, and uses the extracted global motion information to determine a common area between a plurality of image frames. It can be estimated.

The extracting of the common background region may include obtaining a background region for each R, G, and B channel with respect to a common region among the plurality of image frames, and using the background region for each of the R, G, and B channels. A background region for each image frame may be obtained, and a common background region between previous image frames may be extracted based on the current image frame using the background region for each image frame.

The estimating of the light source information may include obtaining brightness components for each of the R, G, and B channels using the extracted background region, and using each of the R, G, and B channels. The brightness component of the appearing light source can be estimated.

Preferably, the estimating of the light source information may obtain a brightness component for each of the R, G, and B channels by averaging the accumulation of edge gradients of each image frame with respect to a common background area of the plurality of image frames.

Preferably, the light source information may include at least one of a brightness component and a color component of the light source appearing in each of the plurality of image frames.

Preferably, the detecting of the color rolling shape may detect a color rolling phenomenon by using the brightness change amount and the color change amount on the time axis which are the estimated light source information.

In addition, the method for detecting color rolling according to the present invention may further include detecting the period and amplitude of the color rolling phenomenon by continuously tracking the detected color rolling phenomenon.

Preferably, the detecting of the period and the amplitude may be performed by detecting a time point at which the accumulation of brightness differences of adjacent image frames for each R, G, and B channel is simultaneously '0', and the brightness obtained within the detected one period. The maximum and minimum values of the components can be used to detect the amplitude.

According to still another aspect of the present invention, there is provided a method for detecting color rolling, comprising: extracting a common background area without motion between a plurality of image frames of an input image sequence; Estimating preset light source information by using the extracted background region; And detecting a color rolling phenomenon by using a change amount of the light source information between the plurality of image frames.

Through this, the present invention estimates the light source information in consideration of the global image change due to the movement of the camera and the local image change due to the movement of the object, and detects the color rolling phenomenon according to the estimated light source information. Can be detected.

In addition, the present invention estimates the light source information in consideration of the global image change due to the movement of the camera and the local image change due to the movement of the object and detects the color rolling phenomenon according to the estimated light source information, thereby improving the accuracy of the color rolling detection. Can be improved.

In addition, the present invention estimates the light source information in consideration of the global image change due to the movement of the camera and the local image change due to the movement of the object, and after detecting the color rolling phenomenon according to the estimated light source information, By estimating, it can use as information for removing the color rolling which generate | occur | produces after detection.

1 is a diagram illustrating an apparatus for detecting color rolling according to an embodiment of the present invention.
2 is a diagram illustrating a result of estimating global motion according to an embodiment of the present invention.
3 is a diagram illustrating a result of extracting a common background region according to an embodiment of the present invention.
4 is a view for explaining the principle of estimating the color temperature according to an embodiment of the present invention.
5 is a diagram illustrating a period and amplitude for each channel due to color rolling according to an embodiment of the present invention.
6 is a diagram illustrating a method for detecting color rolling according to an embodiment of the present invention.

Hereinafter, an apparatus for detecting color rolling and a method thereof according to an embodiment of the present invention will be described with reference to FIGS. 1 to 6. It will be described in detail focusing on the parts necessary to understand the operation and action according to the present invention. Like reference numerals in the drawings denote like elements throughout the specification.

In the present invention, the light source information, for example, the color, the brightness, etc. are estimated in consideration of the global image change due to the camera movement and the local image change due to the object movement, and the color rolling phenomenon is detected according to the estimated light source information. After detecting the rolling phenomenon, we propose a new method to estimate the color rolling period and amplitude.

1 is a diagram illustrating an apparatus for detecting color rolling according to an embodiment of the present invention.

As shown in FIG. 1, an apparatus for detecting color rolling according to the present invention includes an image input unit 110, a motion estimation unit 120, a background region extraction unit 130, a light source information estimation unit 140, and a rolling device. The phenomenon detection unit 150, the period estimation unit 160, and the like may be configured.

The image input unit 110 may receive an image sequence photographed through the image sensor.

The motion estimator 110 may estimate global motion between the plurality of image frames of the input image sequence. That is, the motion estimator 110 extracts global motion information from a current image frame by using a block matching algorithm for a predetermined number of previous image frames.

이라고 하면, 전역 움직임을 제거한 n-k번째 영상 프레임을 다음의 [수학식 1]과 같이 나타낼 수 있다.Based on the nth current video frame, the global motion information shown in the kth previous video frame

In this case, the nk-th image frame from which the global motion is removed may be represented by Equation 1 below.

[Equation 1]

Here, A may represent one of the R, G, and B channels, and m may represent the macro convex.

2 is a diagram illustrating a result of estimating global motion according to an embodiment of the present invention.

As shown in FIG. 2, the result of estimating the global motion between three image frames of the image sequence is shown. The motion estimator 110 estimates a common area between the plurality of image frames by using the extracted global motion information.

For example, a common area of a rectangle of a solid line may be estimated in three image frames. The rectangle of the solid line in the image frame located on the left and the image frame located in the right side is estimated based on the rectangle of the solid line in the center image frame.

Here, the global movement may be a concept including a movement of a camera, that is, a movement including a pan, zoom, rotation, and the like.

The background region extractor 130 may extract a common background region within a region common to a plurality of image frames. The background region extraction unit 130 uses a local motion detection method, and divides an image frame into unit blocks to detect local motion for each unit block and extracts a common background area in order to reduce the amount of computation and reduce the influence of noise.

라고 하면, 각 채널별 배경 영역

을 기 설정된 임계값 Thres_motion를 이용하여 다음의 [수학식 2]와 같이 구할 수 있다.At this time, the brightness average of the m th unit block in the n-1 th image frame

Speaking, background area of each channel

By using the threshold threshold Thres _motion can be obtained as shown in [Equation 2].

&Quot; (2) "

을 다음의 [수학식 3]과 같이 구할 수 있다.Background area for each image frame using the background area of each channel obtained through Equation 2 above

Can be obtained as shown in Equation 3 below.

&Quot; (3) "

Using the background region of each image frame obtained through Equation 3, a common background region M _n between previous frames can be obtained as shown in Equation 4 below based on the current image frame.

&Quot; (4) "

3 is a diagram illustrating a result of extracting a common background region according to an embodiment of the present invention.

As shown in FIG. 3, a result of extracting a common background region between three image frames of an image sequence is shown. The background region extractor 130 may extract a common background region having no movement within a region common to a plurality of image frames.

For example, the common background area is shown in white in the figure.

The light source information estimator 140 may estimate light source information using the extracted common background area. Here, the light source information may include a brightness component, a color or a color temperature component.

Since the change due to the movement between each image frame is removed in the foregoing process, the brightness change due to color rolling can be detected based on the same information for each image frame in estimating the change in the brightness of the time axis. In this case, by estimating the brightness of each image frame for each channel, the brightness and the color change due to color rolling may be detected for each image frame. Therefore, since the brightness of each channel of the light source provides key information in detecting color rolling afterwards, accurate estimation is necessary.

In the present invention, in extracting light source information using information inside an image, a gray edge assumption method may be used. This assumes that the information reflected by the light source at the edge of the object includes brightness and color information and that the sum of edge gradients of the entire image is achromatic.

The light source information estimator 140 obtains an edge gradient of each image frame with respect to the common background region of the n image frames, and extracts the brightness components of the R, G, and B channels as shown in Equation 5 below.

[Equation 5]

Here, N may represent the position of the pixel.

The light source information estimating unit 140 may estimate the brightness component I _n of the light source appearing in each image frame by using the brightness components of each R, G, and B channel as shown in Equation 6 below.

&Quot; (6) "

Here, α ^A may represent specific gravity for each of R, G, and B channels.

In addition, the light source information estimator 140 may estimate a color component or color temperature of a light source appearing in each image frame using, for example, the Robertson method. Referring to this, first, the light source information estimating unit 140 may perform chromaticity conversion as shown in Equation 7 using R, G, and B values of a given light source.

[Equation 7]

Equation 7 may be converted into a chromaticity plane of x _n and y _n as shown in Equation 8 below.

[Equation 8]

In the planckian locus on the chromaticity plane, the color temperature T _n of the current frame can be estimated as shown in Equation 9 below.

&Quot; (9) "

Here, d _l represents the distance between the point reflecting the specification of the image and the line segment representing the l-th same color temperature, and d _{l + 1} represents the line segment representing the point of the image specification and the (l + 1) th same color temperature. It can represent the distance between. And T _l may represent the l-th color temperature on the chromaticity plane.

4 is a view for explaining the principle of estimating the color temperature according to an embodiment of the present invention.

As shown in FIG. 4, in order to estimate the color temperature T _n of the current frame, T _l representing the l-th color temperature and T _{l + 1} representing the (l + 1) -th color temperature, and points reflecting the specificity of the image; The distance d _l to the line segment representing the l-th color temperature, the point reflecting the specificity of the image, and the distance d _{l + 1} to the line segment representing the (l + 1) th color temperature are shown.

The distance between the point and the line segment or the straight line can be obtained by the following Equation 10.

&Quot; (10) "

The rolling phenomenon detection unit 150 may detect the color rolling phenomenon by using the amount of change in brightness and the change in color temperature on the time axis, and may be detected through Equation 11 below.

&Quot; (11) "

Here, Thres ₁ , Thres ₂ , Thres ₃ , and Thres ₄ may represent thresholds for color rolling detection.

The period estimator 160 may detect the period and amplitude of the color rolling phenomenon by continuously tracking the detected color rolling phenomenon. In the present invention, the period and amplitude of the color rolling phenomenon can be detected and used for color rolling correction that occurs later.

5 is a diagram illustrating a period and amplitude for each channel due to color rolling according to an embodiment of the present invention.

As shown in FIG. 5, a light source change for each channel due to color rolling, for example, a color temperature change in FIG. (A), a brightness change in FIG. (B), and a cycle and amplitude of the color rolling phenomenon according to it in FIG. Giving.

When the period of the color rolling phenomenon is P, the brightness component may be expressed by Equation 12 according to the period.

[Equation 12]

The period estimator 160 can obtain a period P through a value satisfying the following Equation 13 using the fact that each of the R, G, and B channels changes independently, but the periods are the same.

&Quot; (13) "

는 R 채널의 인접한 영상 프레임 간의 밝기 차이를 나타내고,

는 G 채널의 인접한 영상 프레임 간의 밝기 차이를 나타내며,

는 B 채널의 인접한 영상 프레임 간의 밝기 차이를 나타낼 수 있다.Where k represents the frame position on the time axis,

Denotes the difference in brightness between adjacent image frames of the R channel,

Indicates the difference in brightness between adjacent video frames in the G channel,

Denotes a difference in brightness between adjacent image frames of the B channel.

That is, the period estimator may obtain a time point at which the accumulation of brightness differences of adjacent image frames for each R, G, and B channel is simultaneously '0'.

When the above Equation 13 is satisfied, the maximum value and the minimum value within one period can be expressed as Equation 14 below.

&Quot; (14) "

The period estimator 160 may estimate the amplitude using the maximum and minimum values thus obtained.

As such, when the color rolling phenomenon is estimated to occur, the color rolling period can be estimated by tracking the same point where the cumulative value of the brightness difference between the image frames of each channel is canceled. The accuracy of color rolling phenomenon detection can be improved. In addition, when a color rolling phenomenon is found, color fluidity of each image frame may be corrected through a color rolling removal algorithm using a period, a maximum value, and a minimum value obtained when color rolling is detected.

6 is a diagram illustrating a method for detecting color rolling according to an embodiment of the present invention.

As shown in FIG. 6, the color rolling detection apparatus (hereinafter referred to as a detection apparatus) according to the present invention may receive an image sequence consisting of a plurality of image frames (S610).

Next, the detection apparatus may estimate global motion between the plurality of image frames of the input image sequence (S620). That is, the detection apparatus extracts global motion information for a predetermined number of previous image frames adjacent to the current image frame by using a block matching algorithm, and estimates a common area between the plurality of image frames using the extracted global motion information. Done.

Next, the detection apparatus may extract a common background area having no motion in a common area between the plurality of image frames (S630). That is, the detection apparatus obtains a background region for each R, G, and B channel, obtains a background region for each image frame using the background region for each R, G, and B channel, and uses a background region for each image frame. The common background region between the previous image frames is extracted based on the current image frame.

Next, the detection apparatus may estimate the light source information by using the extracted common background area (S640). Here, the light source information may include at least one of a brightness component, a color, or a color temperature component.

That is, the detection apparatus obtains the brightness components of each R, G, and B channel using the extracted common background region, and estimates the brightness components of the light source appearing in each image frame using the brightness components of each of the R, G, and B channels. In addition, the color component of the light source appearing in each image frame is estimated by using the extracted common background region.

Next, the detection apparatus may detect the color rolling phenomenon by using the amount of change in the light source information between the plurality of image frames (S650). That is, the detection apparatus detects the color rolling phenomenon by using the brightness change amount and the color change amount on the time axis which are estimated light source information.

Next, the detection apparatus may continuously track the detected color rolling phenomenon to detect the period and amplitude of the color rolling phenomenon (S660). That is, the detection device detects a time point at which the accumulation of brightness differences of adjacent image frames for each R, G, and B channel is '0' at the same time, and detects the maximum and minimum values of brightness components obtained within one detected period. To detect the amplitude.

Various modifications and variations may be made by those skilled in the art to which the apparatus and method for detecting color rolling according to the present invention belong without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate rather than limit the scope of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

110: video input unit
120: motion estimation unit
130: background region extraction unit
140: light source information estimation unit
150: rolling phenomenon detection unit
160: period estimation unit

Claims (20)

A motion estimator for estimating global motion between a plurality of image frames of an input image sequence;
A background region extraction unit configured to extract a common background region without movement between a plurality of image frames by using the estimated global movement;
A light source information estimator configured to estimate preset light source information by using the extracted common background region; And
A rolling phenomenon detection unit detecting a color rolling phenomenon by using a change amount of the light source information between the plurality of image frames;
Apparatus for detecting color rolling comprising a. The method according to claim 1,
The motion estimation unit,
Detecting color rolling comprising extracting global motion information for a predetermined number of previous image frames from a current image frame and estimating a common area between a plurality of image frames using the extracted global motion information. Device for. The method according to claim 1,
The background region extraction unit,
Obtaining a background region for each R, G, and B channel for a common region among the plurality of image frames, and obtaining a background region for each image frame using the background region for each of the R, G, and B channels;
And extracting a common background region between previous image frames based on the current image frame by using the background region for each image frame. The method according to claim 1,
The light source information estimating unit,
The brightness component of each R, G, and B channel is obtained using the extracted common background region, and the brightness component of the light source appearing in each image frame is estimated using the obtained brightness component of each of the R, G, and B channels. A device for detecting color rolling. 5. The method of claim 4,
The light source information estimating unit,
And a brightness component for each of the R, G, and B channels is obtained by averaging the accumulation of edge gradients of each image frame with respect to a common background area of the plurality of image frames. The method according to claim 1,
And the light source information comprises at least one of a brightness component and a color component of a light source appearing in each of the plurality of image frames. The method according to claim 1,
The rolling phenomenon detection unit,
And detecting a color rolling phenomenon by using the estimated brightness change amount and color change amount on the time axis. The method according to claim 1,
A period estimator configured to continuously track the detected color rolling phenomenon and detect a period and an amplitude of the color rolling phenomenon;
The apparatus for detecting color rolling further comprises. The method of claim 8,
The period estimator,
Periodically detects when the accumulation of brightness differences of adjacent video frames for each R, G, and B channel is '0' at the same time, and detects amplitude using the maximum and minimum values of the brightness components obtained within one detected period. Apparatus for detecting color rolling, characterized in that. delete Estimating global motion among a plurality of image frames of an input image sequence;
Extracting a common background region having no motion between a plurality of image frames by using the estimated global motion;
Estimating preset light source information by using the extracted common background area; And
Detecting a color rolling phenomenon by using a change amount of the light source information between the plurality of image frames;
And a method for detecting color rolling. 12. The method of claim 11,
Estimating the global motion,
Detecting color rolling comprising extracting global motion information for a predetermined number of previous image frames from a current image frame and estimating a common area between a plurality of image frames using the extracted global motion information. Way. 12. The method of claim 11,
Extracting the common background region may include:
Obtaining a background region for each R, G, and B channel for a common region among the plurality of image frames, and obtaining a background region for each image frame using the background region for each of the R, G, and B channels;
And extracting a common background region between previous image frames based on a current image frame by using the background region for each image frame. 12. The method of claim 11,
Estimating the light source information,
The brightness component of each R, G, and B channel is obtained using the extracted common background region, and the brightness component of the light source appearing in each image frame is estimated using the obtained brightness component of each of the R, G, and B channels. A method for detecting color rolling. 15. The method of claim 14,
Estimating the light source information,
And a brightness component for each of the R, G, and B channels is obtained by averaging the accumulation of edge gradients of each image frame with respect to a common background area of the plurality of image frames. 12. The method of claim 11,
And the light source information comprises at least one of a brightness component and a color component of a light source appearing in each of the plurality of image frames. 12. The method of claim 11,
Detecting the color rolling phenomenon,
And detecting the color rolling phenomenon by using the estimated brightness change amount and color change amount on the time axis. 12. The method of claim 11,
Continuously tracking the detected color rolling phenomenon to detect a period and amplitude of the color rolling phenomenon;
The method for detecting color rolling further comprising. 19. The method of claim 18,
Detecting the period and amplitude,
Detecting a point in time at which the brightness difference of all adjacent image frames of each R, G, and B channel is all '0' and detecting the amplitude using the maximum and minimum values of the brightness components obtained within one detected period. A method for detecting color rolling. delete

Images