TWI741305B - Image processing method and device - Google Patents

Abstract

The embodiment of the present invention discloses an image processing method and device. The method includes: adjusting the brightness of each frame of image in the video; performing offset compensation for each frame of the image after the brightness adjustment; The pixels of each frame of image that are compensated by the amount are filtered in the time domain. Through the scheme of this embodiment, the brightness or color fluctuations existing in the video are effectively removed.

Description

Image processing method and device

Related applications

The present invention is based on a Chinese patent application with an application number of 201810961926.5 and an application date of 2018/8/22, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated by reference into the present invention.

The embodiment of the present invention relates to image processing technology, in particular to an image processing method and device.

Because the light source is unstable (for example, in daily lighting, since the power source is AC, the brightness of the light source will change with the amplitude of the AC power), the video taken by the camera (including digital cameras, mobile phone cameras, etc.) will appear as the light source changes. The phenomenon of light and dark fluctuations and even color fluctuations from frame to frame. This phenomenon is especially obvious when shooting a video with a high frame rate (greater than or equal to 240 frames per second) when the frame rate (the frame rate used when shooting video is used to distinguish the frame rate used when playing the video) is high.

At present, for a video with a light source frequency of 60 Hz and a frame rate of 240 frames per second, the brightness and color fluctuations of the video can be significantly improved by superimposing and averaging two adjacent frames. In principle, as long as the shooting frame rate is an integer multiple of the light source frequency, the brightness fluctuation phenomenon can be improved by superimposing several adjacent frames (the average brightness of each frame of the image in the video or the brightness of each pixel in the image varies with The phenomenon of light and dark fluctuations due to changes in the number of frames) and color fluctuations (due to the inconsistency of the brightness fluctuations of each color channel, the color changes with the brightness fluctuations).

However, this solution has obvious effects only for videos whose frame rate is an integer multiple of the frequency of the light source, and other video effects are not obvious. And the video processed by this method, when there are fast moving objects when shooting the video, the ghost of the moving objects will appear.

At least some embodiments of the present invention provide an image processing method and device, which can remove brightness or color fluctuations existing in a video.

In one of the embodiments of the present invention, an image processing method is provided, including: Adjust the brightness of each frame of the video; Perform offset compensation for each frame of image after brightness adjustment; Time-domain filtering is performed on the pixels of each frame of image after offset compensation.

In an optional embodiment, the adjusting the brightness of each frame of image in the video includes: The following processing is performed on each frame of image: Count the color averages of the three color channels respectively; the three color channels refer to the red R channel, the green G channel and the blue B channel; The preset first filtering scheme is adopted, and the corresponding color channel is subjected to the first time-domain filtering according to the color average value of each color channel.

In an optional embodiment, said performing offset compensation for each frame of image that has undergone brightness adjustment includes: using a preset offset compensation algorithm to obtain any two adjacent frames of the brightness-adjusted video The offset between the images is compensated for, so that the image content of any two adjacent frames in the video at the same image coordinates remains consistent.

In an optional embodiment, the time-domain filtering of pixels of each frame of image after offset compensation includes: adopting a preset second filtering scheme for each pixel in each frame of video image Perform the second time domain filtering, so that the current frame image and the frame image prior to the current frame are linearly superimposed.

In an optional embodiment, the method further includes: after performing offset compensation for each frame of image that has undergone brightness adjustment, judging whether the image contains a moving object.

In an optional embodiment, the judging whether an image contains a moving object includes: Block the current frame image and the previous frame image respectively to obtain multiple first block images; According to a preset difference calculation algorithm, calculate the difference between the two corresponding first block images in the current frame image and the previous frame image respectively; Comparing the difference between the two first segmented images with a preset difference threshold; When the difference between the two first divided images is greater than or equal to the difference threshold, it is determined that the two first divided images are not similar, and the two first divided images are determined The corresponding image area in the current frame image and the previous frame image contains a moving object; when the difference between the two first segmented images is less than the difference threshold, it is determined that the two The first segmented images are similar, and it is determined that the corresponding image regions of the two first segmented images in the current frame image and the previous frame image do not contain moving objects.

In an optional embodiment, the acquiring the offset between any two adjacent frames of the video after the brightness adjustment includes: Block the current frame image and the previous frame image after brightness adjustment to obtain multiple second block images; Respectively calculating the offset between the two corresponding second block images in the current frame image and the previous frame image; Exclude the second segmented image containing the moving object from the plurality of second segmented images, and use the average value of the offsets of the remaining second segmented images as the current frame image and all Describe the offset between the previous frame of image.

In an optional embodiment, the method further includes: After judging that any frame of image contains a moving object, generate a mask image of the frame of image; wherein, the pixel value of the image area containing the moving object in the mask image is 1 , The pixel value of the image area that does not contain the moving object is 0; According to a preset fusion algorithm, the mask image is used to fuse the corresponding frame image that has been filtered in the time domain with the pixels and the corresponding frame image that has not been filtered in the time domain, so that the The image area containing the moving object is preserved.

In an optional embodiment, the method further includes: performing spatial filtering on each frame of the fused image.

In one of the embodiments of the present invention, there is also provided an image processing device, including: a processor and a computer-readable storage medium, the computer-readable storage medium stores instructions, characterized in that, when the instruction When executed by the processor, the image processing method described in any one of the above is implemented.

The embodiment of the present invention includes: adjusting the brightness of each frame of image in the video; performing offset compensation for each frame of the image after the brightness adjustment; performing offset compensation on the pixels of each frame of the image. Time domain filtering. Through the scheme of this embodiment, the brightness or color fluctuations existing in the video are effectively removed.

Other features and advantages of the embodiments of the present invention will be described in the following description, and partly become obvious from the description, or understood by implementing the present invention. The purpose and other advantages of the embodiments of the present invention can be realized and obtained through the structures specifically indicated in the specification, the scope of the patent application, and the drawings.

In order to make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in detail below in conjunction with the drawings. It should be noted that, in the case of no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other arbitrarily.

The steps shown in the flowchart of the figures can be executed in a computer system such as a set of computer-executable instructions. And, although a logical sequence is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than here.

Example one

An image processing method, as shown in Figure 1, may include steps S101-S103: S101: Perform brightness adjustment on each frame of image in the video; S102: Perform offset compensation for each frame of image that has undergone brightness adjustment; S103: Perform temporal filtering on the pixels of each frame of image that have undergone offset compensation.

In the embodiment of the present invention, for each frame of image in the video, the overall brightness adjustment is performed first, and then image matching is performed, that is, offset compensation is performed on each frame of image that has undergone brightness adjustment; The matched result is filtered in the time domain, which effectively reduces the brightness fluctuation effect of high-speed video shooting under fluctuating light sources, and removes the brightness or color fluctuations in the video.

In the embodiments of the present invention, the specific methods, algorithms, and devices for brightness adjustment, image matching, and time-domain filtering are not limited, and can be implemented by any currently existing method, algorithm, or device according to different application scenarios.

In the embodiment of the present invention, it should be noted that the solution in this embodiment can be applied to a high frame rate (such as a frame rate greater than or equal to 240 frames per second), which may include, but is not limited to: a frame rate greater than or equal to 240 frames Video per second, less than or equal to 960 frames per second. The frequency of the light source may include, but is not limited to: 60HZ and 50HZ. Any video that can adopt the solution of the embodiment of the present invention is within the protection scope of the embodiment of the present invention.

Example two

This embodiment provides a specific implementation of brightness adjustment on the basis of the first embodiment.

In the embodiment of the present invention, as shown in FIG. 2, the adjusting the brightness of each frame of the video may include: performing the following steps S201-S202 on each frame of the image: S201: Count the color averages of the three color channels respectively; the three color channels refer to the red R channel, the green G channel and the blue B channel; S202: Using a preset first filtering scheme, perform first time-domain filtering on the corresponding color channel according to the color average of each color channel.

、

、

。可以採用預設的第一濾波方案，分別對三個顏色通道的顏色均值進行第一時域濾波。In the embodiment of the present invention, for each frame of image, the color averages of the three color channels are respectively counted. It is assumed that the color averages of the three RGB color channels of the current frame image are respectively

,

,

. The preset first filtering scheme may be used to perform the first time domain filtering on the color averages of the three color channels respectively.

In the embodiment of the present invention, the first filtering scheme may include, but is not limited to: a finite impulse response filter or an infinite impulse response filter.

為例解釋如何通過有限脈衝回應濾波器進行時域濾波：假設

為先於當前幀圖像

幀圖像的

通道的顏色均值，對

通道的顏色均值進行時域濾波可以用以下關係式描述：

其中，

為

通道時域濾波的結果，

為濾波係數。令

，用

乘以當前幀圖像中每個像素中

通道的像素值，作為該幀圖像中每個像素中

通道的像素值。In the embodiment of the present invention, here is

Take an example to explain how to perform time-domain filtering through a finite impulse response filter: hypothesis

Is prior to the current frame of image

Frame image

The color average of the channel, right

The time-domain filtering of the color mean value of the channel can be described by the following relationship:

in,

for

The result of channel time domain filtering,

Is the filter coefficient. make

,use

Multiply by each pixel in the current frame of image

The pixel value of the channel, as each pixel in the frame of the image

The pixel value of the channel.

顏色通道的顏色均值進行時域濾波，並利用時域濾波後的結果去調整每個像素的每個顏色通道的像素值，從而實現對視頻中的每一幀圖像進行亮度調整，在此不再一一贅述。In the embodiment of the present invention, similar operations are performed on other channels (G channel and B channel) to realize the calculation of each pixel in the current frame image.

The color average value of the color channel is filtered in the time domain, and the result of the time domain filtering is used to adjust the pixel value of each color channel of each pixel, so as to realize the brightness adjustment of each frame of the video. Let me repeat them one by one.

Example three

This embodiment provides a specific implementation of image matching on the basis of Embodiment 1 or Embodiment 2.

In the embodiment of the present invention, the offset compensation for each frame of the image that has undergone brightness adjustment may include: using a preset offset compensation algorithm to obtain any two adjacent frames of the brightness-adjusted video The offset between the images is compensated for, so that the image content of any two adjacent frames in the video at the same image coordinates remains consistent.

In the embodiment of the present invention, due to the shaking of the camera (for example, the shaking of the hand when holding the camera can cause the shaking of the camera), the current frame image will have a certain offset relative to the previous frame image, which causes two problems. The content of the images under the same image coordinates in the frame images are inconsistent, and this phenomenon will have an adverse effect on the subsequent image temporal filtering. In addition to the offset, there may be camera rotation, but if only two adjacent frames of images are considered, the camera rotation can be ignored. The purpose of image matching is to find the offset between images, and to eliminate the inconsistency of image content through offset compensation.

In the embodiment of the present invention, the consistency of the image content of any two adjacent frames of images located at the same image coordinates in the video may mean: between any two adjacent frames of images, any two images Regions with the same content are at the same position under the same image coordinates, or in other words, the deviation of the position is less than or equal to a preset deviation threshold.

In the embodiment of the present invention, the offset compensation algorithm may include, but is not limited to: a template matching algorithm and/or a matching algorithm based on feature points. The two offset compensation algorithms will be described in detail below.

In the embodiment of the present invention, the simplest offset compensation algorithm may be a classic template matching algorithm. The basic principle can be briefly described as follows: intercept an area of the same size as the template image in the reference image, and use it as the intercepted image. Compare the template image with the intercepted image, and calculate the difference between them. Among them, the technical indicators for evaluating image differences may include, but are not limited to: normalized cross-correlation, average absolute difference, error sum of squares, and absolute error sum, etc. You can manually set the range of the starting position of the captured image in the reference image, calculate the difference between all captured images and the template image in this range, and use the starting position of the captured image corresponding to the smallest difference as the offset between the template image and the reference image. quantity.

In the embodiment of the present invention, the basic principle of the matching algorithm based on feature points can be simply described as follows: extract feature points from the two images to be matched, and obtain feature point matching pairs through the feature point matching algorithm. The feature point matching pair calculates the offset between the two images.

In the embodiment of the present invention, there are many algorithms for extracting feature points. For example, they may include, but are not limited to, the classic SIFT algorithm (SIFT, that is, Scale-invariant feature transform), HARRIS algorithm, and the like.

In the embodiment of the present invention, the algorithm for feature point matching may include, but is not limited to, the SIFT algorithm, the SURF (Speeded Up Robust Features) algorithm, and the like.

In the embodiment of the present invention, in addition to calculating the offset between the two images according to these feature point matching pairs, the optical flow of the feature points can also be obtained (for example, using the classic Lucas-Kanade algorithm), and by excluding Abnormal optical flow (a simple method can be to set a threshold, optical flow greater than or equal to this threshold is regarded as abnormal optical flow, optical flow less than this threshold is regarded as non-abnormal optical flow), and the remaining features The optical flow of the points is averaged as the offset of the entire image.

Example four

In this embodiment, on the basis of the third embodiment, in order to eliminate the influence of the moving object on the offset compensation process, another specific implementation manner for obtaining the offset of the image is given.

In the embodiment of the present invention, as shown in FIG. 3, the obtaining the offset between any two adjacent frames of the video after the brightness adjustment may include steps S301-S303: S301: Separately block the current frame image and the previous frame image after brightness adjustment, to obtain multiple second block images; S302: Calculate the offsets between the two corresponding second block images in the current frame image and the previous frame image respectively; S303: Excluding the second segmented image containing the moving object from the plurality of second segmented images, and use the average value of the offsets of the remaining second segmented images as the current frame image The offset between the image and the previous frame.

In the embodiment of the present invention, in order to eliminate the influence of the moving object on the offset compensation, the image may be divided into blocks first, and the offset corresponding to each image block (that is, the second block image mentioned above) can be calculated separately. Then exclude those image blocks that are affected by moving objects, and average the offset of the remaining image blocks as the offset of the entire image.

Example five

On the basis of the third or fourth embodiment, this embodiment provides a specific implementation manner for further time-domain filtering of the video after offset compensation.

In the embodiment of the present invention, the time-domain filtering of pixels of each frame of image after offset compensation may include: adopting a preset second filtering scheme to each pixel in each frame of video image Perform the second time domain filtering, so that the current frame image and the image prior to the current frame are linearly superimposed.

In the embodiment of the present invention, the time-domain filtering in this step (ie, the second time-domain filtering) is similar to the time-domain filtering (ie, the first time-domain filtering) in step S101, except that this step is performed for each pixel. Do time domain filtering.

In the embodiment of the present invention, when it needs to be explained, the first time-domain filtering and the second time-domain filtering are only used to distinguish the time-domain filtering in two different steps, and are not used to limit the specific scheme and the second time-domain filtering. Implementation order, etc. The first time domain filtering and the second time domain filtering may adopt the same or different time domain filtering schemes.

In the embodiment of the present invention, the second filtering scheme may include, but is not limited to: a finite impulse response filter or an infinite impulse response filter.

其中，

為當前幀圖像，

為先於當前幀圖像

幀的圖像，

為視頻的第一幀圖像，

為時域濾波後的結果，

為圖像像素座標，

為濾波係數。In the embodiment of the present invention, the following takes the filtering of the finite impulse response filter as an example for detailed description. Specifically, filtering through a finite impulse response filter can be achieved through the following relationship:

in,

Is the current frame image,

Is prior to the current frame of image

Frame image,

Is the first frame of the video,

Is the result of time domain filtering,

Is the image pixel coordinates,

Is the filter coefficient.

In the embodiment of the present invention, time-domain filtering is performed on each frame image in the video, and the filtered result is a linear superposition of the current frame image and the historical frame image (that is, the image prior to the current frame).

Example Six

This embodiment is based on the third or fourth embodiment, in order to protect the moving objects in the video from being blurred or ghosting, and for the smooth implementation of the four solutions in the above-mentioned embodiments, the corresponding diagrams are given. An example scheme for judging whether the image contains a moving object, and a specific implementation is given.

In the embodiment of the present invention, the method may further include: after performing offset compensation on each frame of image that has undergone brightness adjustment, determining the moving object in the image.

In the embodiment of the present invention, as shown in FIG. 4, the judging whether an image contains a moving object may include steps S401-S404: S401: block the current frame image and the previous frame image respectively to obtain multiple first block images; S402: According to a preset difference calculation algorithm, respectively calculate the difference between the two corresponding first block images in the current frame image and the previous frame image; S403: Compare the difference between the two first segmented images with a preset difference threshold; S404: When the difference between the two first block images is greater than or equal to the difference threshold, determine that the two first block images are not similar, and determine the two first block images The image contains a moving object in the corresponding image area of the current frame image and the previous frame image; when the difference between the two first segmented images is less than the difference threshold, it is determined The two first block images are similar, and it is determined that the two first block images in the current frame image and the corresponding image area of the previous frame image do not contain motion object.

In the embodiment of the present invention, before comparing the current frame image with the previous frame image, first divide the current frame image and the previous frame image into blocks, and calculate the current frame image in each small block The difference between the image and the previous frame (the difference can be obtained by but not limited to: normalized cross-correlation, average absolute difference, error sum of squares, and absolute error sum algorithms), and a threshold can be preset It can be judged whether the two frames of images are similar in this small block, wherein when the difference is greater than or equal to the threshold, it can be judged as non-similar, otherwise it can be judged as similar. If the judgment result is not similar, it can be considered that the area contains moving objects, otherwise it can be judged that the area does not contain moving objects.

In the embodiment of the present invention, it should be noted that the first segmented image in this embodiment and the second segmented image in the foregoing embodiment are only two different names or labels, mainly for distinguishing purposes. The segmented images obtained in two segmentation operations for different purposes are used to avoid confusion, and there is no distinction in attributes such as order and size.

In the embodiment of the present invention, it should be noted that this step can be performed after offset compensation is performed on each frame of image that has undergone brightness adjustment, or it can be performed after offset of each frame of image that has undergone brightness adjustment. Before the compensation, there are no detailed restrictions on the specific implementation time and sequence.

Example Seven

In this embodiment, on the basis of the sixth embodiment, in order to protect the moving objects in the video from being blurred or ghosting, a further specific implementation manner is given.

In the embodiment of the present invention, as shown in FIG. 5, the method may further include steps S501-S502: S501: After judging that any frame of image contains a moving object, generate a mask image of the frame of image; wherein, the mask image contains the pixel value of the image area of the moving object Is 1, the pixel value of the image area that does not contain the moving object is 0; S502: According to the preset fusion algorithm, use the masked image to compare the corresponding frame image after the time domain filtering (ie, the second time domain filtering) on the pixels and the time domain filtering on the pixels without time domain filtering ( That is, the corresponding frame images of the second temporal filtering) are fused, so that the image area containing the moving object is preserved.

，該遮罩版圖像

為二值圖像，其中，有運動物體的區域的像素值為1，沒有運動物體的區域的像素值為0。In the embodiment of the present invention, when any frame of image is determined to contain a moving object, a masked image of the moving image can be generated

, The masked image

It is a binary image, where the pixel value of the area with moving objects is 1, and the pixel value of the area without moving objects is 0.

將經過時域濾波後的相應幀圖像與未對像素作過時域濾波的相應幀圖像進行融合，以使得包含有運動物體的圖像區域得到保留，從而消除了時域濾波對運動物體造成的模糊、重影等現象。In the embodiment of the present invention, according to the result of the judgment of the moving object, the mask image can be used

The corresponding frame image after time-domain filtering is fused with the corresponding frame image without time-domain filtering of pixels, so that the image area containing moving objects is preserved, thereby eliminating the effect of time-domain filtering on moving objects. Blur, ghosting and other phenomena.

In the embodiment of the present invention, the corresponding frame image after the time-domain filtering (ie, the second time-domain filtering) of the pair of pixels, and the corresponding frame image without the time-domain filtering of the pixels, all refer to The frame image corresponding to the above-mentioned mask image.

可以採用下述的關係式實現圖像融合：

其中，

為融合後的結果，

為圖像時域濾波的結果，

為當前幀圖像，

為圖像的座標。In the embodiment of the present invention, according to the mask image

The following relationship can be used to achieve image fusion:

in,

Is the result of the fusion,

Is the result of image temporal filtering,

Is the current frame image,

Is the coordinates of the image.

In the embodiment of the present invention, the current frame image after time domain filtering can be simply superimposed with the current frame image without time domain filtering through the above relationship, that is, the area containing moving objects adopts the current frame image The pixel value of the result obtained after time domain filtering is used for the area that does not contain moving objects.

In the embodiment of the present invention, the preset fusion algorithm may further include: the classic Laplacian pyramid fusion algorithm.

、

…

，其中

為原圖，後面的每層圖像

都是通過對上一層圖像

進行卷積模糊並下採樣生成的，比如

是由

進行卷積模糊並下採樣獲得，卷積模糊核通常採用高斯核，所以這一系列圖像也稱作高斯金字塔。為了簡單起見這裡用高斯金字塔來表示模糊圖像

、

…

序列，儘管有時模糊圖像序列並非由高斯模糊核產生。假設拉普拉斯金字塔標記為

、

…

，那麼拉普拉斯金字塔每一層的圖像可以由該等式獲得：

，其中

函數可以理解為上採樣。即拉普拉斯金字塔的每一層都是由高斯金字塔中的該層對應的圖像減去高斯金字塔中的下一層的圖像經過上採樣的圖像。值得注意的是最後一層

。通過拉普拉斯金字塔重構圖像是上述過程的逆過程。所以通過拉普拉斯金字塔進行融合的步驟描述如下： 1、對時域濾波後的結果

和當前幀

建立拉普拉斯金字塔分別為

和

。 2、對遮罩版圖像

建立高斯金字塔，記為

。 3、構建新的拉普拉斯金字塔

：

。其中

為金字塔層數下標，為正整數，

為圖像的座標。 4、通過

重構圖像得到結果圖。In the embodiment of the present invention, the basic principle of the classic Laplacian pyramid fusion algorithm is as follows: First, a series of blurred images are generated

,

…

,in

For the original image, each layer of images behind

Through the previous layer of image

Convolutional blur and down-sampling generated, such as

By

Convolutional blur is performed and down-sampling is performed. The convolutional blur kernel usually uses a Gaussian kernel, so this series of images is also called a Gaussian pyramid. For the sake of simplicity, a Gaussian pyramid is used to represent the blurred image.

,

…

Sequence, although sometimes the blurred image sequence is not produced by the Gaussian blur kernel. Suppose the Laplacian pyramid is marked as

,

…

, Then the images of each layer of the Laplace Pyramid can be obtained by the equation:

,in

The function can be understood as upsampling. That is, each layer of the Laplacian pyramid is an image obtained by subtracting the image of the next layer in the Gaussian pyramid from the image corresponding to the layer in the Gaussian pyramid after being up-sampled. It is worth noting that the last layer

. The reconstruction of the image through the Laplacian pyramid is the inverse process of the above process. Therefore, the steps of fusion through the Laplacian pyramid are described as follows: 1. The result after filtering in the time domain

And the current frame

The establishment of the Laplace Pyramid is

with

. 2. For the masked image

Build a Gaussian pyramid, denoted as

. 3. Build a new Laplace pyramid

:

. in

Is the subscript of the number of pyramid levels, which is a positive integer,

Is the coordinates of the image. 4. Pass

Reconstruct the image to get the result map.

Example eight

In this embodiment, on the basis of the seventh embodiment, in order to remove the residual image remaining after the above steps, a further specific implementation manner is given.

In the embodiment of the present invention, the method may further include: performing spatial filtering on each frame of the fused image.

In the embodiment of the present invention, temporal filtering describes the filtering between frames, while spatial filtering refers to filtering a single frame image, and the main purpose is to remove the residual image left after the above steps.

In the embodiment of the present invention, the spatial filtering method may include, but is not limited to: an edge preserved filter; for example, a guided filter, a bilateral filter, etc., filters the image after fusion The result is filtered in the spatial domain of the image to get the final result.

Example 9

On the basis of any of the foregoing embodiments, this embodiment provides a specific implementation manner for processing video images on the basis of video image reduction.

，

縮小後的圖像標記為

。然後對

進行圖像對齊（即進行偏移量補償）、第二時域濾波、運動物體判斷、圖像融合、空域濾波（這些步驟同實施例一到八），將結果記為

。由此可以求得差異圖

：

，其中

為圖像的座標。然後將差異圖

放大到與

同尺寸，得到放大後的差異圖

，將

疊加到

得到最終結果

，其中，

為圖像的座標：

In the embodiment of the present invention, the main flow of the ninth embodiment is basically the same as that of the first to eighth embodiments, and the main difference is that most of the operations in the ninth embodiment are performed on a small diagram. The flow chart is shown in Figure 6. The details are as follows: after the overall brightness adjustment, the image is reduced, and the image after the overall brightness adjustment is marked as

,

The reduced image is marked as

. Then right

Perform image alignment (ie, offset compensation), second time domain filtering, moving object judgment, image fusion, and spatial filtering (these steps are the same as those in Examples 1 to 8), and record the result as

. From this, the difference map can be obtained

:

,in

Is the coordinates of the image. Then the difference map

Zoom in to

The same size, get the enlarged difference map

,will

Superimposed to

Get the final result

,in,

Is the coordinates of the image:

In the embodiment of the present invention, most of the processing is performed on the reduced image, and then the difference between the processed result and the small image before the processing is enlarged and applied to the large image. The solution of this embodiment guarantees the effect Under the premise of, the computing time can be greatly reduced.

Example ten

An image processing device 1, as shown in FIG. 7, includes a processor 11 and a computer-readable storage medium 12, the computer-readable storage medium 12 stores instructions, wherein, when the instructions are received by the processor When 11 is executed, the image processing method described in any one of the above embodiments is implemented.

The embodiment of the present invention includes: adjusting the brightness of each frame of image in the video; performing offset compensation for each frame of the image after the brightness adjustment; performing offset compensation on the pixels of each frame of the image. Time domain filtering. Through the scheme of this embodiment, the brightness or color fluctuations in the video are effectively removed, and moving objects are well preserved, so that they are not blurred and have no afterimages or ghosting.

The technology disclosed in the embodiments of the present invention can be applied to static images, moving images (such as video), and can be applied to any suitable type of image processing device, such as digital cameras, mobile phones, and integrated digital cameras. Electronic equipment, security or video surveillance systems, medical imaging systems, etc.

Combination with hardware

Those with ordinary knowledge in the technical field to which the present invention pertains can understand that all or some of the steps in the method disclosed above, the functional modules/units in the system, and the device can be implemented as software, firmware, hardware, and appropriate combination. In hardware implementations, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be defined by Several physical components are executed in cooperation. Some or all of the components can be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit, such as a dedicated integrated circuit. Such software can be distributed on computer-readable media, and computer-readable media can include computer storage media (or non-transitory media) and communication media (or temporary media). As known to those with ordinary knowledge in the technical field to which the present invention pertains, the term computer storage media includes easy implementations in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Loss and non-volatile, removable and non-removable media. Computer storage media include but are not limited to RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, floppy disk storage or other magnetic Storage device, or any other medium that can be used to store desired information and that can be accessed by a computer. In addition, as is well known to those with ordinary knowledge in the technical field to which the present invention pertains, communication media usually include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and Can include any information delivery media.

1‧‧‧Image processing device 11‧‧‧Processor 12‧‧‧Computer readable storage media S101~S103‧‧‧Step S201~S202‧‧‧Step S301~S303‧‧‧Step S401~S404‧‧‧Step S501~S502‧‧‧Step

The drawings are used to provide a further understanding of the technical solution of the present invention, and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present invention, and do not constitute a limitation to the technical solution of the present invention.

Fig. 1 is a flowchart of an image processing method according to an embodiment of the present invention.

FIG. 2 is a flowchart of a method for adjusting the brightness of each frame of video in an embodiment of the present invention.

FIG. 3 is a flowchart of a method for obtaining the offset between any two adjacent frames of images in a video that has undergone brightness adjustment according to an embodiment of the present invention.

Fig. 4 is a flowchart of a method for judging whether an image contains a moving object according to an embodiment of the present invention.

FIG. 5 is a flowchart of a method for image fusion of an image containing a moving object according to an embodiment of the present invention.

Fig. 6 is a flowchart of a method for processing a video image on the basis of video image reduction according to an embodiment of the present invention.

FIG. 7 is a block diagram of the composition of an image processing device according to an embodiment of the present invention.

S101~S103‧‧‧Step

Claims (10)

An image processing method, including the following steps: adjust the brightness of each frame of image in the video; after the brightness adjustment, perform the reduction processing on each frame of the image; perform the reduction processing on each frame of the reduced image after the brightness adjustment Offset compensation; and time-domain filtering is performed on the pixels of each frame of the reduced image after offset compensation. The image processing method according to claim 1, wherein the adjusting the brightness of each frame of image in the video includes: performing the following processing on each frame of image: separately counting the three color channels Color average; the three color channels refer to the red R channel, the green G channel, and the blue B channel; the preset first filtering scheme is adopted, and the corresponding color channel is filtered in the first time domain according to the color average of each color channel. . The image processing method according to claim 1, wherein said performing offset compensation on each reduced image frame after brightness adjustment includes: adopting a preset offset compensation algorithm to obtain the brightness adjusted The offset between any two adjacent frames in the video, and the offset is compensated so that any two adjacent frames in the video are located at the same image coordinates. The image content remains consistent. The image processing method according to claim 1, wherein the time-domain filtering of pixels of each frame of the reduced image after offset compensation includes: adopting a preset second filtering scheme for each frame Each pixel in the video image is filtered in the second time domain, so that the current frame image and the frame image prior to the current frame are linearly superimposed. The image processing method according to claim 3, wherein the method further includes: After the offset compensation is performed on the reduced image of each frame that has undergone brightness adjustment, it is determined whether the image contains a moving object. The image processing method according to claim 5, wherein the judging whether the image contains a moving object includes: separately dividing the current frame image and the previous frame image to obtain a plurality of first divisions Block image; according to a preset difference calculation algorithm, respectively calculate the difference between the two first block images in the current frame image and the previous frame image; compare the two first block images The difference between the divided images is compared with a preset difference threshold; when the difference between the two first divided images is greater than or equal to the difference threshold, it is determined that the two first divided images are not Similar, and it is determined that the corresponding image regions of the two first block images in the current frame image and the previous frame image contain moving objects; when the two first block images When the difference between the images is less than the difference threshold, it is determined that the two first segmented images are similar, and it is determined that the two first segmented images are between the current frame image and the previous The corresponding image area in the frame image does not contain moving objects. The image processing method according to claim 3, wherein the obtaining the offset between any two adjacent frames of the video that has undergone brightness adjustment includes: separately adjusting the current frame image after brightness adjustment Perform block division with the previous frame of image to obtain multiple second block images; respectively calculate the distance between the two corresponding second block images in the current frame image and the previous frame image的Offset; Exclude the second block image containing the moving object from the plurality of second block images, and use the average value of the offset of the remaining second block images as the current The offset between the previous frame image and the previous frame image. The image processing method according to claim 6, wherein the method further comprises: generating a mask image of the frame image after determining that a moving object is contained in any frame image; wherein The pixel value of the image area that contains the moving object in the mask image is 1, and the pixel value of the image area that does not contain the moving object is 0; according to the preset fusion algorithm, the pixel value is 0; For the masked image, the corresponding frame image after the time-domain filtering of the pixels is fused with the corresponding frame image without the time-domain filtering of the pixels, so that the image containing the moving object The area is reserved. The image processing method according to claim 8, wherein the method further comprises: performing spatial filtering on each frame of the fused image. An image processing device, comprising: a processor and a computer-readable storage medium, the computer-readable storage medium stores instructions, and is characterized in that, when the instructions are executed by the processor, the request item 1 The image processing method described in any one of to 9.

Images