KR102453392B1 - Method and Apparatus for Video Denoising Considering Moving Object - Google Patents

Abstract

Disclosed are a method for removing image noise considering a moving object and an apparatus therefor.
Image noise is reduced by determining the camera movement between frames of the image, estimating the background motion and moving object motion based on the camera movement, and comparing the flow profile for each background motion and moving object motion in the area where the moving object is present with the reference frame. It relates to a method for removing image noise in consideration of a moving object to be removed, and an apparatus therefor.

Description

Method and Apparatus for Video Denoising Considering Moving Object

An embodiment of the present invention relates to a method for removing image noise in consideration of a moving object and an apparatus therefor.

The content described in this section merely provides background information on the embodiments of the present invention and does not constitute the prior art.

In general, image noise removal techniques may be divided into a method of removing image noise in a spatial domain and a method of removing image noise in a time domain.

As a technique for removing noise from an image in a spatial domain, a method of performing filtering in the spatial domain is mainly used. Methods for performing filtering in the spatial domain include a method of removing noise in consideration of a pattern of image noise (Korean Patent Application Laid-Open No. 10-2015-014259) and a method of removing noise in consideration of sensor noise characteristics (Korean Patent Application Laid-Open No. 10-2007-0061309) and the like.

In addition, the technique of removing noise from an image in the time domain is a method of removing image noise by setting a reference image and adjusting the filtering intensity according to the degree of screen change determined using the reference image and the surrounding images (Korean Patent Laid-Open Patent Publication) 10-2004-0024888) is mainly used.

However, since the above-described methods remove image noise by using only a change in background pixels without considering the motion of the camera or the moving object, distortion is generated even when the filtering intensity is adjusted. Therefore, there is a need for a method for accurately removing image noise in consideration of the motion of the camera or the motion of the moving object.

An embodiment of the present invention determines the camera movement between frames of an image, estimates the background motion and the moving object motion based on the camera movement, and uses the flow profile for each of the background motion and the moving object motion in an area where the moving object exists as a reference frame. A main object of the present invention is to provide a method for removing image noise in consideration of a moving object that removes image noise as compared to the present invention, and an apparatus for the same.

According to an aspect of an embodiment of the present invention, in an apparatus for removing image noise in consideration of a moving object, scene change information of an input image is extracted, and the divided image is divided by the same scene based on the scene change information. an input image processing unit to generate; a motion estimator for checking a motion of a camera in the divided image and estimating a background motion and a moving object motion based on the motion of the camera; and an image noise removing unit that removes image noise in a region where a moving object exists by using a flow profile for the background motion and the moving object motion.

Further, according to another aspect of the embodiment of the present invention, in a method of removing image noise in consideration of a moving object, scene change information of an input image is extracted, and the image is divided and divided for the same scene based on the scene change information an input image processing process for generating an image; a motion estimation process of confirming a motion of a camera in the divided image and estimating a background motion and a moving object motion based on the motion of the camera; and an image noise removal process of removing image noise in an area in which a moving object exists by using a flow profile for the background motion and the moving object motion.

As described above, according to the embodiment of the present invention, it is possible to obtain a high-quality image noise removal result regardless of the movement of the camera sensor or the pattern of image noise.

In addition, according to an embodiment of the present invention, as the image noise is removed using partial image information on the moving object region, the time for removing the image noise can be shortened, and there is an effect applicable to a streaming environment.

In addition, according to an embodiment of the present invention, since the image noise removal technology can be applied to a streaming environment, there is an effect that can be applied to various fields such as IPTV, real-time streaming content, and real-time monitoring of an intelligent image system.

1 is a block diagram schematically showing an image noise removing apparatus according to an embodiment of the present invention.
2 is a flowchart illustrating a method for removing image noise according to an embodiment of the present invention.
3 is an exemplary diagram for explaining an operation of removing image noise according to an embodiment of the present invention.
4 is an exemplary diagram for explaining an operation of estimating motions of a background and a moving object in the image noise removing apparatus according to an embodiment of the present invention.
5 is an exemplary diagram for explaining an operation of matching an image and removing image noise in the image noise removing apparatus according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically showing an image noise removing apparatus according to an embodiment of the present invention.

The image noise removing apparatus 100 according to the present embodiment includes an input image processing unit 110 , a motion estimation unit 120 , and an image noise removing unit 130 .

The input image processing unit 110 acquires an input image, determines whether the input image has a scene change, and divides the input image into segments for the same space. Here, the input image is a video image captured by a camera, and may be an image obtained by photographing a plurality of spatial regions. For example, the input image processing unit 110 may acquire an image obtained by photographing a plurality of preset areas using a Pan-Tilt-Zoom (PTZ) camera as the input image.

The input image processing unit 110 extracts scene change information for the input image, and divides the image into images captured in a continuous space using the scene change information. The input image processing unit 110 describes an image segmented based on the same space as a segmented image, and in this embodiment, noise is removed using a frame included in the segmented image.

The input image may be implemented as one split image when there is no spatial change, that is, no scene change, and may be implemented as a plurality of split images when there is a spatial change.

The input image processing unit 110 determines a scene change of the input image by using the scene change information, and specifically converts a frame showing a sudden change based on the scene change information including a color histogram and a feature point correspondence accuracy to a frame in which the scene is changed. By selecting a frame, it is possible to determine whether to change the scene.

Meanwhile, the input image processing unit 110 may determine whether or not a scene has changed based on scene change information included in metadata information of the input image, but is not limited thereto. Difference) It is also possible to determine whether to change a scene by selecting a frame having an error equal to or greater than a threshold. For example, assuming that the input image is an image consisting of 100 frames obtained by photographing area A and area B, the input image processing unit 110 extracts the 50th frame from which area B is photographed based on the scene change information. In addition, the first frame to the 49th frame may be divided into a first divided image, and the 50th frame to the 100th frame may be divided into a second divided image.

The motion estimator 120 estimates a background motion and a moving object motion from the divided image. The motion estimator 120 determines whether the camera moves in the divided image, and estimates a background motion and a moving object motion based on whether the camera moves. The motion estimation unit 120 includes a camera motion determination unit 122 , a background motion estimation unit 124 , and a moving object motion estimation unit 126 .

The camera motion determining unit 122 determines whether there is a camera motion in the divided image. Methods of estimating motion in the background motion estimator 124 and the moving object motion estimator 126 based on the determination result of the camera motion may be changed.

The camera movement determining unit 122 determines the camera movement between frames in the divided image by using the brightness of the pixel. For example, the camera motion determining unit 122 calculates the ratio of pixels showing a predefined difference in brightness or pixels without a difference in brightness from the total number of pixels based on the difference image between frames, and the ratio is less than or equal to a preset threshold. In this case, it is determined that the camera is in a stationary state. Meanwhile, the camera movement determining unit 122 determines that there is movement of the camera when the corresponding ratio exceeds a preset threshold.

Meanwhile, the camera motion determining unit 122 may detect the feature points of each frame and determine the camera movement between frames in the divided image by using the feature point correspondence between the frames. For example, the camera motion determining unit 122 calculates the ratio of the movement amount corresponding to the feature point showing a movement below a certain threshold from the total number of feature point correspondence relationships. judge to be On the other hand, the camera movement determining unit 122 determines that there is movement of the camera when the ratio of the movement amount corresponding to the feature point exceeds a predetermined threshold.

The camera motion determining unit 122 may simplify the motion estimation operations of the background motion estimation unit 124 and the moving object motion estimation unit 126 when there is no or little camera motion. That is, when the camera motion determining unit 122 determines that the camera between frames is in a stationary state, the background motion estimating unit 124 and the moving object motion estimating unit 126 estimate the characteristic point correspondence result as a moving object motion, and the same between frames Regions can be treated as background motion. The camera motion determining unit 122 does not need to separately estimate the grid-based background motion in the background motion estimation unit 124 when the camera between frames is stationary.

The background motion estimation unit 124 estimates a background motion for a static background between frames in the divided image. The background motion estimation unit 124 may estimate the background motion based on the camera motion determination result of the camera motion determination unit 122 .

When there is no camera movement between frames in the divided image, the background motion estimator 124 estimates a motionless portion, that is, a pixel area without a brightness difference, a feature point portion without motion, etc. as the background motion. In the case of a stationary state in which there is no camera movement, the moving object motion estimation unit 126 may estimate a moving part, that is, a pixel area having a brightness difference equal to or greater than a threshold value, a moving feature point part, etc. as the moving object motion.

On the other hand, when there is camera movement between frames in the divided image, the background motion is estimated in the following way.

The background motion estimation unit 124 estimates the background motion by dividing the frames in the divided image into grids and estimating the motion of the entire frame by collecting motions for each grid. For example, the background motion estimator 124 divides the segmented image into a predefined grid, and performs regression analysis on homography for each segmented grid using the feature point correspondence relationship F. to estimate the background motion. Here, the feature point correspondence relationship F may extract a feature point using a Scale Invariant Feature Transform (SIFT) method and estimate the background motion by analyzing it using a RANdom SAmple Consensus (RANSAC) method. In the case of using the RANSAC method, the background motion estimation unit 124 estimates the background motion by removing the characteristic point correspondence relationship in which the reprojection error is equal to or greater than a threshold value, that is, the characteristic point correspondence relationship generated from the moving object.

Preferably, the background motion estimation unit 124 extracts a feature point using the SIFT method and estimates the background motion by analyzing it using the RANSAC method, but is not limited thereto. If the background motion can be estimated based on a grid, various methods can be applied. The background motion estimator 124 may use a method of selecting an average motion based on a grid after matching feature points between frames, or may estimate motion with respect to the background by propagating the motion based on the grid and applying predetermined spatial filtering. For example, the background motion estimation unit 124 divides the image into a predefined grid, then propagates the feature point correspondence to the vertices of each grid, and performs two-dimensional filtering on each vertex and the image space. It is possible to estimate the background motion with the moving object removed.

After estimating the background motion, the moving object motion estimation unit 126 estimates the moving object motion with respect to a frame included in the divided image. Specifically, the moving object motion estimation unit 126 estimates the motion (M) of the moving object by using the difference between the key point correspondence information (F) and the background motion (B) generated based on the key point correspondence relationship. Here, the moving object motion estimation unit 126 may estimate the moving object motion in the same grid method as the background motion estimation method. The moving body motion estimation unit 126 may estimate the moving body motion using [Equation 1].

The moving body motion estimation unit 126 may estimate the moving body motion by propagating the difference between the key point correspondence and the background motion to the periphery or using an average value of the difference between the key point correspondence and the background motion at a specific location. For example, the moving object motion estimation unit 126 performs 2D filtering by propagating the difference between the feature point correspondence and the background motion to the peripheral vertices of the grid to estimate the grid-based moving object motion, or calculates the average motion value around the grid vertices. It is possible to estimate the motion of the moving body by calculating it. Here, the moving body motion estimator 126 may estimate the moving body motion by applying a penalty function to the difference between the feature point correspondence and the background motion to suppress a minute difference and highlight a large difference.

The moving object motion estimator 126 may extract a moving object motion equal to or greater than a preset threshold among the moving object motions, and set a grid including the corresponding moving object motion as the moving object area.

The image noise removing unit 130 performs an operation of removing noise from an image based on the estimated background motion and the moving object motion. The image noise removing unit 130 includes an image matching unit 132 , a profile estimating unit 134 , and a filtering unit 136 .

The image matching unit 132 sets a window including a plurality of frames in the divided image, and matches the reference frame based on the background motion of the plurality of frames included in the window. Specifically, the image matching unit 132 sets a reference frame in a sliding window method in the divided image, and sets a plurality of frames included in the window using a pre-estimated background motion to coordinates of the reference frame. sort (match) with

The image matching unit 132 includes n previous frames (t-1, t-2, ..., t-n) and n subsequent frames (t+1, t+2, .. ., t+n) is used to match the reference frame (t). Here, the registration is preferably processed based on Galilean transformations, but is not necessarily limited thereto. The image matching unit 132 preferably matches the background motion to the reference frame t by performing inverse transformation in the case of n subsequent frames, and the background motion up to a specific position may be calculated by summing the inter-frame motions.

Since the image matching unit 132 cannot accurately distinguish the region for the motion of the moving object, the image matching of the reference frame is processed using only the background motion between frames.

The profile estimator 134 classifies the moving body region for the moving body motion in the frame included in the window, and since the moving body and the background are mixed in the moving body region, the flow profile for the moving body motion and the background motion is respectively estimated. . Here, the flow profile refers to information obtained by collecting brightness values according to time at a specific pixel position in frames within the window.

As the profile estimator 134 estimates both the flow profiles for the moving object and the background in the moving object region, the filtering unit 136 may simultaneously consider the flow profiles for the moving object motion and the background motion to perform image noise removal. On the other hand, the profile estimator 134 estimates only the flow profile for the background motion in the region where only the background motion exists in the reference frame.

The profile estimator 134 may remove noise in the flow profile for the moving object or the background. Specifically, the profile estimator 134 assumes that the distribution of values for color, brightness, etc. included in the flow profile is a normal distribution, removes outliers that deviate from a predetermined standard deviation coefficient, and excludes the removed values. A flow profile can be generated by estimating the mean and variance of the remaining values. For example, the profile estimator 134 assumes that the distribution of values included in the flow profile is a Gaussian distribution, removes a value deviating from a predetermined sigma coefficient (eg, 3 sigma), and averages the remaining values. and estimating variance to generate a flow profile.

The profile estimator 134 may remove noise in the flow profile by assuming a normal distribution for each of the flow profiles for the moving object and the background.

The filtering unit 136 filters the flow profile of the moving object or background, and removes image noise through the filtering process.

When image noise is removed using only the background flow profile for the background region, the filtering unit 136 applies filtering (1d filtering) to the background flow profile because it assumes that all pixels corresponding to the background have the same physical location. Can be applied to remove image noise.

On the other hand, when the background area and the moving object area are mixed, the filtering unit 136 calculates a similarity by comparing each of the background flow profile and the moving object flow profile with the reference frame, and filters the high similarity flow profile (1d filtering). Remove image noise.

Conventionally, noise is removed by using only background motion in a reference frame, and when the moving object area is small, there is not much change in color, brightness, etc., so it is not a problem. However, when the moving object is large or moving quickly, accurate noise removal is impossible. Accordingly, the filtering unit 136 according to the present exemplary embodiment may compare each of the background flow profile and the moving object flow profile with the reference frame to efficiently remove image noise according to the degree of similarity.

The filtering unit 136 according to the present embodiment compares the average value of each of the background flow profile and the moving object flow profile with the pixel value of the reference frame, selects a flow profile with a small difference, that is, a high similarity, and filters the selected flow profile. Remove image noise. Here, the filtering may apply a Gaussian filter, a Bilateral filter, a Median filter, etc., but is not necessarily limited thereto, and any filter that can be used for blurring or smoothing of an image is changed to various types of filters Applicable.

The filtering unit 136 calculates the background similarity between the background flow profile and the pixels of the reference frame using [Equation 2] in the moving object region, and using [Equation 3], the moving object between the moving object flow profile and the pixels of the reference frame Calculate the similarity.

: 기준 프레임 인덱스, t: 분할 영상 인덱스(

-W ≤ t ≤

+W), P_B: 배경 플로우 프로파일, P_M: 이동체 플로우 프로파일, P_B ^E: Edge 영상에서 배경의 edgeness 프로파일 및 P_M ^E: Edge 영상에서 이동체의 edgeness 프로파일)(S _B : background similarity, S _M : moving object similarity, x: pixel position, W: window size,

: reference frame index, t: segmented image index (

-W ≤ t ≤

+W), P _B : background flow profile, P _M : moving object flow profile, P _B ^E : background edgeness profile in edge image, and P _M ^E : moving object edgeness profile in edge image)

The filtering unit 136 calculates the background similarity and the moving object similarity using [Equation 2] and [Equation 3], and selects a flow profile having a high similarity among the two similarities. The filtering unit 136 generates an image noise removal result obtained by removing image noise by applying filtering (1d filtering) to the selected flow profile. Here, the filtering unit 136 is described as applying the filtering to the selected flow profile, but the present invention is not limited thereto. Instead of filtering, an average value of values in the flow profile may be applied to remove image noise. Here, the filtering unit 136 determines that the similarity is small as the temporal distance from the reference frame increases, the color is different, and the edgeness difference increases. The smaller the edgeness difference, the higher the similarity.

2 is a flowchart illustrating a method for removing image noise according to an embodiment of the present invention.

The image noise removing apparatus 100 acquires an input image photographed using a camera (S210). Here, the input image is a video image captured by a camera, and may be an image obtained by photographing a plurality of spatial regions.

The image noise removing apparatus 100 extracts scene change information for the input image (S212), and divides the input image by the same spatial region using the scene change information to generate a divided image (S220). Here, the scene change information includes a color histogram, feature point correspondence accuracy, etc., and the image noise removal apparatus 100 selects a frame showing a sudden change based on the scene change information as a scene changed frame to generate a divided image. .

The image noise removing apparatus 100 determines the movement of the camera between frames in the divided image (S230). The image noise removing apparatus 100 detects a feature point of each frame, and determines the camera movement between frames in the divided image using the feature point correspondence relationship between frames, or a predefined value from the total number of pixels based on the difference image between frames. The movement of the camera can be determined by calculating the ratio of pixels showing a difference in brightness or pixels without a difference in brightness.

When there is no movement of the camera in step S230, the image noise removal apparatus 100 initializes a mesh flow for the movement of the background between frames to estimate the background motion (S232), and sets the remaining area with the movement to the moving object. It is estimated by motion (S250). Here, the mesh flow for the motion of the background means a set of two-dimensional vectors in the form of a grid for a non-moving part due to no motion, that is, a pixel region without a difference in brightness, a feature point part without motion, and the like.

On the other hand, if there is camera movement in step S230, the image noise removing apparatus 100 estimates the background motion by dividing the frames in the divided image into grids, and estimating the motion of the entire frame by collecting the motions for each grid. (S240), the motion of the moving object is estimated using the difference between the keypoint correspondence information generated based on the keypoint correspondence relationship and the background motion (S250).

The image noise removal apparatus 100 performs an operation of estimating the background motion and the moving object motion in chronological order for a plurality of frames included in the segmented image, and when motion estimation of all frames in the segmented image is completed (S252), the segmented image A predetermined window is set for the window sliding process ( S260 ).

The image noise removing apparatus 100 matches the reference frame based on the background motion (S270). Specifically, the image noise removing apparatus 100 sets a reference frame in a sliding window method in the divided image, and sets a plurality of frames included in the window using the pre-estimated background motion to coordinate the reference frame. ) to sort (match).

The image noise removing apparatus 100 estimates flow profiles for the moving object motion and the background motion, respectively ( S280 ). Specifically, the image noise removing apparatus 100 provides information about the moving object motion in a frame included in the window. The moving object area is divided, and since the moving object and the background are mixed in the moving object area, flow profiles for the moving object motion and the background motion are respectively estimated. Here, the flow profile refers to information obtained by collecting brightness values according to time at a specific pixel position in frames within the window.

The image noise removal apparatus 100 filters the flow profile to remove image noise (S290), and generates an image noise removal result (S292). Specifically, when the image noise removal apparatus 100 performs image noise removal on the background region, the image noise removal apparatus 100 removes the image noise by applying filtering (1d filtering) to the background flow profile.

On the other hand, when the image noise removal apparatus 100 performs image noise removal on the moving object region, the background flow profile and the moving object flow profile are respectively compared with the pixel values of the reference frame to calculate the similarity, and a flow profile with high similarity is obtained. The image noise is removed by filtering.

Although it is described that each process is sequentially executed in FIG. 2 , it is not necessarily limited thereto. In other words, since it may be applicable to changing and executing the process described in FIG. 2 or executing one or more processes in parallel, FIG. 2 is not limited to a time-series order.

The method for removing image noise according to the present embodiment illustrated in FIG. 2 may be implemented as an application (or program) and recorded in a recording medium readable by a terminal device (or computer). A recording medium in which an application (or program) for implementing the method for removing image noise according to the present embodiment is recorded and a terminal device (or computer) readable recording medium is any type of recording device in which data readable by a computing system is stored includes

3 is an exemplary diagram for explaining an operation of removing image noise according to an embodiment of the present invention.

The image noise removing apparatus 100 divides and processes the divided image 310 for the same space based on the acquired scene change information of the input image.

The image noise removing apparatus 100 estimates a motion 320 for each of a plurality of frames included in the divided image 310 . The image noise removing apparatus 100 divides each frame in the divided image 310 into a grid, and estimates a grid-based background motion and a moving object motion, respectively.

The image noise removing apparatus 100 performs image registration by using the background motion of frames included in the window of the divided image 310 . That is, the image noise removing apparatus 100 sets a reference frame using the background motion, and generates a flow profile for each of the background motion and the moving object motion based on the reference frame.

The image noise removing apparatus 100 removes (330) image noise by filtering one or both of the background flow profile and the moving object flow profile.

When only the background flow profile exists, the image noise removing apparatus 100 filters the background flow profile to remove the image noise. However, when there is a moving object flow profile, since the moving object and the background are mixed in the moving object area, the similarity is calculated by comparing the background flow profile and the moving object flow profile with the pixels of the reference frame, and filtering the flow profile with high similarity. Remove image noise.

4 is an exemplary diagram for explaining an operation of estimating motions of a background and a moving object in the image noise removing apparatus according to an embodiment of the present invention.

The left image of FIG. 4A shows the result of the feature point correspondence between frames in the divided image, and the right image of FIG. 4A shows the result of estimating the background motion based on the grid. The image noise removing apparatus 100 may estimate the background motion by dividing the frames in the divided image into grids, and estimating the motion of the entire frame by collecting motions for each grid. For example, the image noise removal apparatus 100 divides the frame into grids, then propagates the feature point correspondence to the vertices of each grid, and removes the moving object by performing two-dimensional filtering on each vertex and the image space. Background motion can be estimated.

The left image of FIG. 4(b) represents an image of the divided image, and the right image of FIG. 4(b) represents the result of estimating the motion of the moving object based on the grid. The image noise removal apparatus 100 divides the frames in the divided image into grids, and propagates the difference between the feature point correspondence and the background motion in each grid to the periphery, or uses the average value of the difference between the feature point correspondence and the background motion at a specific location to calculate the motion of the moving object. can be estimated For example, the image noise removal apparatus 100 performs 2D filtering by propagating the difference between the feature point correspondence and the background motion to the peripheral vertices of the grid in order to estimate the grid-based motion of the moving object, or calculates the average motion value around the grid vertices. It is possible to estimate the motion of the moving body by calculating it.

5 is an exemplary diagram for explaining an operation of matching an image and removing image noise in the image noise removing apparatus according to an embodiment of the present invention.

5A is an exemplary diagram of setting a reference frame by performing image registration using a grid-based background motion. As shown in (a) of FIG. 5 , the image noise removing apparatus 100 is configured among five frames t-2, t-1, t, t+1, and t+2 included within a preset window size. The background motion of some frames (t-2, t-1, t+1, t+2) is matched with the reference frame (t).

FIG. 5B is an exemplary diagram illustrating an operation of selecting a flow profile based on a similarity between a background flow profile and a moving object flow profile in order to remove image noise from the moving object area. As shown in (b) of FIG. 5 , the image noise removing apparatus 100 extracts a moving object region from a frame included in the window, and compares each of the background flow profile and the moving object flow profile of the moving object region with the pixels of the reference frame. to calculate the similarity. The image noise removing apparatus 100 may remove image noise by selecting a flow profile having a high similarity, that is, a flow profile having a minimum error among a background flow profile and a moving object flow profile.

The above description is merely illustrative of the technical spirit of the embodiment of the present invention, and those of ordinary skill in the art to which the embodiment of the present invention pertains may modify various modifications and transformation will be possible. Therefore, the embodiments of the present invention are not intended to limit the technical spirit of the embodiment of the present invention, but to explain, and the scope of the technical spirit of the embodiment of the present invention is not limited by these embodiments. The protection scope of the embodiment of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the embodiment of the present invention.

As described above, the embodiment of the present invention is applied to the field of removing image noise, so that high-quality image noise removal results can be obtained regardless of the movement of the camera sensor or the pattern of image noise, and partial It is a useful invention that produces an effect of reducing the time for removing image noise by using image information to remove image noise.

100: image noise canceling device
110: input image processing unit 120: motion estimation unit
122: camera motion determining unit 124: background motion estimation unit
126: moving body motion estimation unit
130: image noise removing unit 132: image matching unit
134: profile estimator 136: filtering unit

Claims (9)

An apparatus for removing image noise in consideration of a moving object, the apparatus comprising:
an input image processing unit for extracting scene change information of an input image, and dividing the image for the same scene based on the scene change information to generate a divided image;
a motion estimator for checking a motion of a camera in the divided image and estimating a background motion and a moving object motion based on the motion of the camera; and
Estimating a background flow profile based on the background motion, estimating a moving object flow profile based on the moving object motion, and using the background flow profile and the moving object flow profile to measure image noise in an area where the moving object is present Image noise canceling unit
Image noise removal device comprising a. According to claim 1,
The image noise removing unit,
an image matching unit for matching frames in a window with a reference frame using the background motion;
a profile estimator for estimating the background flow profile and the moving object flow profile; and
A filtering unit that removes image noise by comparing pixel values of the reference frame with each of the background flow profile and the moving object flow profile in an area where the moving object is present
Image noise removal device comprising a. 3. The method of claim 2,
The filtering unit,
Comparing each of the background flow profile and the moving object flow profile with the pixel value of the reference frame to calculate each similarity, and filtering by selecting a flow profile having a high similarity among the background flow profile and the moving object flow profile Image noise canceling device. According to claim 1,
The motion estimation unit,
The image noise removing apparatus according to claim 1, wherein the movement of the camera is confirmed by comparing a difference in brightness between a plurality of frames included in the divided image or a feature point correspondence ratio for movement with a preset threshold. 5. The method of claim 4,
The motion estimation unit,
When there is no movement of the camera, the pixel without the brightness difference or the feature point without the motion is estimated as the background motion, and the pixel with the brightness difference is estimated as the moving object motion. . 5. The method of claim 4,
The motion estimation unit,
When there is movement of the camera, the frames in the divided image are divided into grids, the motions for each grid are combined to estimate the background motion, and the difference between the feature point correspondence information between the frames and the background motion is used to control the movement of the moving object. An apparatus for removing noise from an image, characterized in that for estimating motion. A method of removing image noise in consideration of a moving object, the method comprising:
an input image processing process of extracting scene change information of an input image and dividing the image for the same scene based on the scene change information to generate a divided image;
a motion estimation process of confirming a motion of a camera in the divided image and estimating a background motion and a moving object motion based on the motion of the camera; and
Estimating a background flow profile based on the background motion, estimating a moving object flow profile based on the moving object motion, and using the background flow profile and the moving object flow profile to measure image noise in an area where the moving object is present Image noise removal process
Image noise removal method comprising a. 8. The method of claim 7,
The image noise removal process is
an image matching process of matching frames in a window with a reference frame using the background motion;
a profile estimation process of estimating the background flow profile and the moving object flow profile; and
A filtering process of removing image noise by comparing pixel values of the reference frame with each of the background flow profile and the moving object flow profile in the region where the moving object is present
Image noise removal method comprising a. 9. The method of claim 8,
The filtering process is
Comparing each of the background flow profile and the moving object flow profile with the pixel value of the reference frame to calculate each similarity, and filtering by selecting a flow profile having a high similarity among the background flow profile and the moving object flow profile A method for removing noise from images.

Images