WO2023236757A1 - 视频图像噪声评估方法及装置 - Google Patents
视频图像噪声评估方法及装置 Download PDFInfo
- Publication number
- WO2023236757A1 WO2023236757A1 PCT/CN2023/095180 CN2023095180W WO2023236757A1 WO 2023236757 A1 WO2023236757 A1 WO 2023236757A1 CN 2023095180 W CN2023095180 W CN 2023095180W WO 2023236757 A1 WO2023236757 A1 WO 2023236757A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- noise
- video image
- area
- foreground
- image
- Prior art date
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 13
- 238000000034 method Methods 0.000 claims abstract description 53
- 238000005259 measurement Methods 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000013135 deep learning Methods 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 230000000007 visual effect Effects 0.000 claims description 8
- 238000003708 edge detection Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 abstract description 18
- 230000019771 cognition Effects 0.000 abstract description 5
- 238000010586 diagram Methods 0.000 description 10
- 230000000694 effects Effects 0.000 description 5
- 238000010606 normalization Methods 0.000 description 4
- 230000008447 perception Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Definitions
- the embodiments of the present application relate to the technical field of noise identification, and in particular, to a video image noise assessment method and device.
- noise estimation there is a solution for noise estimation from the perspective of signal filtering.
- the noise estimation in this method is rough and unstable. In actual applications, the effect is limited and it takes too long.
- the method used to determine the noise is also not detailed enough, and the noise it determines cannot be processed later for local noise reduction, and if the deep learning model used is to achieve satisfactory results, it usually requires a large number of fine annotations.
- sufficient and completely annotated large amounts of video noise data usually do not exist and require a lot of effort to complete high-quality annotation, so the implementation process is relatively complex.
- Embodiments of the present application provide a video image noise assessment method and device, which can achieve precise positioning of noise points, accurately assess the noise level of video images, and provide good support for subsequent directional noise reduction processing.
- embodiments of the present application provide a video image noise assessment method, which method includes:
- the whole-image noise information of the video image, as well as the foreground noise information corresponding to the foreground area and the background noise information corresponding to the background area are determined according to the noise data.
- embodiments of the present application also provide a video image noise assessment device, including:
- a pixel noise determination module configured to perform noise estimation on the video image to obtain noise data for each pixel in the video image
- An image area dividing module configured to divide the video image into areas to obtain a foreground area and a background area
- a noise information determination module configured to determine, based on the noise data, the full-image noise information of the video image, as well as the foreground noise information corresponding to the foreground area and the background noise information corresponding to the background area.
- embodiments of the present application also provide a video image noise assessment device, which includes:
- processors one or more processors
- a storage device for storing one or more programs
- the one or more processors are caused to implement the video image noise assessment method described in the embodiments of this application.
- embodiments of the present application also provide a storage medium that stores computer-executable instructions, which when executed by a computer processor are used to perform the video image noise assessment method described in the embodiments of the present application. .
- embodiments of the present application also provide a computer program product.
- the computer program product includes a computer program.
- the computer program is stored in a computer-readable storage medium.
- At least one processor of the device reads the computer program from the computer-readable storage medium.
- Obtain and execute the computer program causing the device to execute the video image noise assessment method described in the embodiment of the present application.
- the noise data of each pixel in the video image is obtained by performing noise estimation on the video image, the video image is divided into regions to obtain the foreground area and the background area, and the full-image noise information of the video image is determined based on the noise data.
- this noise evaluation method is more refined, down to the noise of the pixels, and at the same time divides the area and evaluates the noise information individually for each area. A more purposeful noise level estimate that is more consistent with subjective cognition can be obtained, providing good support for subsequent directional noise reduction processing.
- Figure 1 is a flow chart of a video image noise assessment method provided by an embodiment of the present application
- Figure 2 is a schematic diagram of a visual display of a noise map provided by an embodiment of the present application.
- Figure 3 is a schematic diagram of overlaying video image noise onto the original video image provided by an embodiment of the present application
- Figure 4 is a flow chart of a video image noise assessment method including transformation and adjustment of input images provided by an embodiment of the present application;
- Figure 5 is a flow chart of a video image noise assessment method including edge processing of video images provided by an embodiment of the present application
- Figure 6 is a flow chart of a video image noise assessment method for regional division of video images provided by an embodiment of the present application
- Figure 7 is a schematic diagram of dividing a video image into a foreground area and a background area according to an embodiment of the present application
- Figure 8 is a flow chart of a video image noise assessment method for determining full-image noise information provided by an embodiment of the present application
- Figure 9 is a flow chart of a video image noise assessment method for determining noise information in the foreground area and background area provided by an embodiment of the present application.
- Figure 10 is a structural block diagram of a video image noise assessment device provided by an embodiment of the present application.
- Figure 11 is a schematic structural diagram of a video image noise assessment device provided by an embodiment of the present application.
- Figure 1 is a flow chart of a video image noise assessment method provided by an embodiment of the present application. It can be used to perform noise assessment on video images or single images to determine the noise situation of the image.
- This method can be performed by computing devices such as servers, smart terminals, Laptops, tablets, etc. are used for execution. Taking the server as the execution device as an example, it can be a Linux server for video back-end processing. The specific steps include the following:
- Step S101 Perform noise estimation on the video image to obtain noise data for each pixel in the video image.
- video image noise assessment is performed through an integrated algorithm module, and noise estimation is performed on the input video image to obtain noise data for each pixel in the video image.
- the video image may be an image of a live video, or an input separate static image, etc.
- the noise data can be the noise value corresponding to the pixel point, that is, for each pixel point in the video image, the specific noise value of the pixel point is obtained.
- the noise value ranges from 0 to 1, where a larger value indicates that the noise is more obvious.
- the noise estimation process for the video image is: performing noise estimation on the video image through the trained multi-layer deep learning neural network to obtain the noise data of each pixel in the video image.
- the multi-layer deep learning neural network includes a plurality of stacked residual network modules.
- the multi-layer deep learning neural network outputs a noise map of the same size as the input video image.
- the noise map can be represented and stored in the form of a matrix. Each element in the matrix corresponds to a pixel in the video image, and the value of a specific element is the noise value of the pixel.
- noise is estimated on the video image through a multi-layer deep learning neural network, and after the noise map is output, the noise map is visually displayed.
- Figure 2 is a schematic diagram of a visual display of a noise map provided by an embodiment of the present application.
- the bright spot area in the figure is the noise part in the video image. The more concentrated and brighter the point corresponding to The noise in the video image is more obvious.
- the noise points can be superimposed on the original image for display.
- Figure 3 is a schematic diagram of overlaying video image noise onto the original video image provided by an embodiment of the present application, so that the video image noise can be observed more intuitively.
- Step S102 Divide the video image into areas to obtain a foreground area and a background area.
- the video image when performing noise estimation on a video image, is divided into regions to obtain a foreground region and a background region.
- the purpose of area division is to avoid uneven noise due to the vignette effect, and the area where the user's eyes focus on viewing is limited and usually concentrated in the central area.
- the video image is further divided into regions to obtain a foreground area and a background area, where the foreground area is the divided area where the user's eyes are focused on viewing, and the background area is the divided area where the user's eyes are not focused on viewing.
- partitioned noise statistics are performed based on the divided areas.
- the foreground area can be an area surrounded by rectangles, circles, and other graphics of fixed size framed with the video image as the center.
- the area of the video image outside the enclosed area is the background area.
- Step S103 Determine the full-image noise information of the video image according to the noise data, as well as the foreground noise information corresponding to the foreground area and the background noise information corresponding to the background area.
- the divided foreground area and background area, foreground noise information and background area, as well as the whole image noise information are determined respectively, and the statistics of the three types of noise information are used to determine Truly reflects the noise level of each input video image.
- the full-image noise information represents the noise situation of the entire video image
- the foreground noise information represents the noise situation in the areas that the human eye pays attention to
- the background noise information represents the noise situation in the areas that are not focused by the human eye. That is, the noise situation in different areas is evaluated based on the determined noise data to reflect the real noise situation that conforms to the subjective perception of the human eye. For example, for the same noise value, if the noise appears in the foreground area relative to the background area, the worse the video image quality, the more serious the noise situation.
- the information of the video image containing the three variables is output respectively, for example, in the form of an intuitive graph, to Display the noise level in different areas.
- the noise data of each pixel in the video image is obtained by performing noise estimation on the video image, the video image is divided into regions to obtain the foreground area and the background area, and the full-image noise information of the video image is determined based on the noise data, and The foreground noise information corresponding to the foreground area and the background noise information corresponding to the background area.
- This noise evaluation method is more refined, down to the noise of pixels. At the same time, it divides the area and evaluates the noise information separately for each area. Obtain a noise level estimate that is more purposeful and more consistent with subjective cognition, providing good support for subsequent directional noise reduction processing.
- Figure 4 is a flow chart of a video image noise assessment method that includes transformation and adjustment of input images provided by an embodiment of the present application. As shown in Figure 4, it specifically includes:
- Step S201 When it is determined that the resolution of the input video image is smaller than the preset resolution size, edge padding is performed on the video image to obtain the video image of the preset resolution size, and the adjusted video image is Normalization of image pixel values.
- noise evaluation processing is performed on video images of a fixed size to improve processing efficiency.
- the set default resolution size is 1280x720 (720P).
- edge filling is performed on the video image to obtain a video with the preset resolution size.
- image For example, the black edge area of the input video image is filled to obtain a video image of a fixed size.
- the adjusted video image is normalized to the image pixel value, and the original pixel value range is adjusted from [0, 255] to the [0, 1] range to facilitate the processing of pixel noise data. determination to improve computing efficiency.
- the recording mode includes a horizontal recording mode and a vertical recording mode. Assuming that the input video image set for video image noise assessment is in the vertical mode, then If it is detected that the current video image is in non-vertical mode, it will be adjusted to vertical mode. For example, assuming that the current recording mode of the video image is the horizontal mode, the video image is rotated 90° to adjust to the vertical mode.
- the input video image when the resolution of the input video image is higher than the set preset resolution size, the input video image may be cropped or scaled down to obtain and set the fixed size. size of the video image.
- Step S202 Perform noise estimation on the video image to obtain noise data for each pixel in the video image.
- Step S203 Divide the video image into areas to obtain a foreground area and a background area.
- Step S204 Determine the full-image noise information of the video image according to the noise data, as well as the foreground noise information corresponding to the foreground area and the background noise information corresponding to the background area.
- Figure 5 is a video image including edge processing on the video image provided by the embodiment of the present application.
- the flow chart of the noise assessment method is shown in Figure 5, which specifically includes:
- Step S301 When it is determined that the resolution of the input video image is smaller than the preset resolution size, edge padding is performed on the video image to obtain the video image of the preset resolution size, and the adjusted video image is Normalization of image pixel values.
- Step S302 Perform noise estimation on the video image after resolution adjustment and normalization to obtain noise data for each pixel in the video image.
- Step S303 Restoring the size and image pixel value of the adjusted video image, performing edge detection on the restored video image to obtain high-frequency edge information, and eliminating the high-frequency edge information.
- the process of restoring the size of the adjusted video image may be to delete the area filled during edge filling, that is, to restore the original image size, which is consistent with the actual video recording or image shooting. size, optimizing the visual display effect of noise assessment.
- edge detection is performed on the restored video image to obtain high-frequency edge information, and the high-frequency edge information is eliminated.
- the edge detection method for the restored video image can be to use the edge detection Canny algorithm to separate the high-frequency edge information potentially existing in the noise data from the noise signal, that is, to remove the high-frequency edges. In order to avoid the impact on the determination of the noise value when the subsequent noise information is determined.
- Step S304 Perform area division on the video image after restoration processing and high-frequency edge removal to obtain a foreground area and a background area.
- Step S305 Determine the full-image noise information of the video image according to the noise data, as well as the foreground noise information corresponding to the foreground area and the background noise information corresponding to the background area.
- Figure 6 is a flow chart of a video image noise assessment method for regional division of video images provided by an embodiment of the present application. As shown in Figure 6, it specifically includes:
- Step S401 Perform noise estimation on the video image to obtain noise data for each pixel in the video image.
- Step S402 Construct an inscribed ellipse with the center of the video image as the origin, determine the image area where the inscribed ellipse is located as the foreground area, and determine the image area outside the inscribed ellipse as the background area.
- an inscribed ellipse when dividing the video image area, is constructed with the center of the video image as the origin, where the area size of the inscribed ellipse is determined according to the size of the video image and preset adjustment parameters.
- the constructed ellipse can be expressed as:
- parameters H and W are the height and width of the video image respectively, and ratio is the preset adjustment parameter.
- the size of the preset adjustment parameter is set to meet different observation needs and different specific business scenarios. For example, it is set to 0.8.
- Figure 7 is a schematic diagram of dividing a video image into a foreground area and a background area provided by an embodiment of the present application, in which 4021 is the foreground area and 4022 is the background area.
- Step S403 Determine the full-image noise information of the video image according to the noise data, as well as the foreground noise information corresponding to the foreground area and the background noise information corresponding to the background area.
- Figure 8 is a flow chart of a video image noise assessment method for determining full-image noise information provided by an embodiment of the present application. As shown in Figure 8, it specifically includes:
- Step S501 Perform noise estimation on the video image to obtain noise data for each pixel in the video image.
- Step S502 Divide the video image into regions to obtain a foreground region and a background region.
- Step S503 Obtain the average weight threshold calculated in advance based on the image data set, filter the noise values of the pixels in the noise data that are greater than the average weight threshold, and average the noise values of the filtered pixels to obtain the video. Full image noise information.
- the average measurement threshold is first calculated and analyzed based on the image data set.
- the average measurement threshold may be, for example, a value calculated through statistical analysis. That is, when the noise value of a pixel is greater than the average measurement threshold, it is relatively obvious noise.
- the noise values determined for each pixel in the input video image are filtered, That is, points with relatively obvious noise values are screened out, and the noise values of the screened pixels are averaged to obtain the full image noise information of the video image.
- f weighted is the calculated weight value
- x is the brightness value of each pixel
- a, b and c are the parameters of the specific function equation.
- the value range of a can be [1.5, 3]
- the value range of b can be [30, 50]
- the value range of c can be [1.5, 3].
- the image data set may be an image data set containing the entire video of the video image. For example, taking the value range of the noise value as [0, 1], the average measurement threshold is determined to be 0.76 through calculation.
- the average measurement threshold is determined to be 0.76 through calculation.
- Step S504 Determine the foreground noise information corresponding to the foreground area and the background noise information corresponding to the background area according to the noise data.
- the noise value of each pixel is used for calculation.
- the obvious noise values are screened, and the filtered noise values are calculated using a weighted average method to obtain the noise information of the whole image, where , the weight value is calculated based on the brightness value of the pixel, so that the final calculated noise information is more in line with the subjective perception of the human eye and the noise evaluation effect is better.
- Figure 9 is a flow chart of a video image noise assessment method for determining noise information in the foreground area and background area provided by the embodiment of the present application. As shown in Figure 9, it specifically includes:
- Step S601 Perform noise estimation on the video image to obtain noise data for each pixel in the video image.
- Step S602 Divide the video image into regions to obtain a foreground region and a background region.
- Step S603 Determine the full-image noise information of the video image according to the noise data.
- Step S604 Obtain the average weight threshold calculated in advance based on the image data set, and filter the noise values of the pixels in the noise data of the foreground area and the background area that are greater than the average weight threshold to obtain the foreground noise value and background noise value.
- the average measurement threshold is first calculated and analyzed based on the image data set.
- the average measurement threshold may be, for example, a value calculated through statistical analysis. That is, when the noise value of a pixel is greater than the average measurement threshold, it is relatively obvious noise.
- the noise values determined for each pixel in the input video image are filtered, that is, the points with relatively obvious noise values are filtered out, and the foreground area and background are obtained respectively.
- Step S605 Calculate the foreground noise value proportion based on the foreground noise value and the area size of the foreground area, and calculate the background noise value proportion based on the background noise value and the area size of the background area.
- the method of determining the foreground noise information and the background noise information may be to determine the proportion of noise values. Specifically, for the foreground area, the sum of the noise values filtered out in the foreground area is calculated and divided by the area size of the foreground area to obtain the proportion of the foreground noise value. In the same way, for the background area, calculate the sum of the noise values filtered out in the background area, and divide it by the area size of the background area to obtain the background noise value ratio, that is, the noise value ratio represents the noise in the corresponding area divided by the noise value ratio. Condition.
- the noise values of the pixels filtered out in the foreground area are respectively noise 1 to noise i , and then the weight value corresponding to each noise value is calculated respectively.
- the weight value is calculated based on the brightness value of the corresponding pixel point.
- the weight values corresponding to the noise values noise 1 to noise i are respectively recorded as weigt 1 to weigt i .
- the noise value of each pixel is used for calculation.
- the obvious noise values are screened, and the filtered noise values are calculated using a weighted average to obtain the regional noise information to characterize Noise evaluation under different subjective perceptions of the human eye, in which the weight value is calculated based on the brightness value of the pixel, making the final calculated noise information more consistent with the subjective perception of the human eye, and the noise assessment effect is better.
- Figure 10 is a structural block diagram of a video image noise assessment device provided by an embodiment of the present application. It is configured to execute the video image noise assessment method provided in the above embodiment, and has functional modules and beneficial effects corresponding to the execution method. As shown in Figure 10, the device specifically includes: a pixel noise determination module 101, an image area division module 102 and a noise information determination module 103, where,
- the pixel noise determination module 101 is configured to perform noise estimation on the video image to obtain noise data for each pixel in the video image;
- the image area dividing module 102 is configured to divide the video image into areas to obtain a foreground area and a background area;
- the noise information determination module 103 is configured to determine the full-image noise information of the video image according to the noise data, as well as the foreground noise information corresponding to the foreground area and the background noise information corresponding to the background area.
- the noise data of each pixel in the video image is obtained by performing noise estimation on the video image, the video image is divided into regions to obtain the foreground area and the background area, and the full-image noise information of the video image is determined based on the noise data, and The foreground noise information corresponding to the foreground area and the background noise information corresponding to the background area.
- This noise evaluation method is more refined, down to the noise of pixels. At the same time, it divides the area and evaluates the noise information separately for each area. Obtain a noise level estimate that is more purposeful and more consistent with subjective cognition, providing good support for subsequent directional noise reduction processing.
- the pixel noise determination module is configured as:
- the multi-layer deep learning neural network includes a plurality of stacked residual network modules
- a visual noise map corresponding to the video image is generated based on the noise data.
- the device further includes an image processing module configured to:
- the image processing module is configured as:
- edge detection is performed on the restored video image to obtain high-frequency edge information, and the high-frequency edge information is eliminated.
- the image area dividing module is configured as:
- an inscribed ellipse is constructed, and the area size of the inscribed ellipse is determined according to the size of the video image and the preset adjustment parameters;
- the image area where the inscribed ellipse is located is determined as the foreground area, and the image area outside the inscribed ellipse is determined as the background area.
- the noise information determination module is configured as:
- the noise values of the filtered pixels are averaged to obtain the full image noise information of the video image.
- the noise information determination module is configured as:
- the mean value is calculated to obtain the full image noise information of the video image.
- the noise information determination module is configured as:
- the foreground noise value ratio is calculated based on the foreground noise value and the area size of the foreground area
- the background noise value ratio is calculated based on the background noise value and the area size of the background area.
- the noise information determination module is configured as:
- Figure 11 is a schematic structural diagram of a video image noise evaluation device provided by an embodiment of the present application.
- the device includes a processor 201, a memory 202, an input device 203 and an output device 204; the processor 201 in the device The number can be one or more.
- one processor 201 is taken as an example; the processor 201, memory 202, input device 203 and output device 204 in the device can be connected through the bus. Or connect in other ways, Figure 11 takes connection through bus as an example.
- the memory 202 can be used to store software programs, computer-executable programs and modules, such as program instructions/modules corresponding to the video image noise assessment method in the embodiment of the present application.
- the processor 201 executes software programs, instructions and modules stored in the memory 202 to execute various functional applications and data processing of the device, that is, to implement the above video image noise assessment method.
- the input device 203 may be used to receive input numeric or character information and generate key signal inputs related to user settings and functional control of the device.
- the output device 204 may include a display device such as a display screen.
- Embodiments of the present application also provide a storage medium containing computer-executable instructions, which when executed by a computer processor are used to perform a video image noise assessment method described in the above embodiments, which includes:
- the whole-image noise information of the video image, as well as the foreground noise information corresponding to the foreground area and the background noise information corresponding to the background area are determined according to the noise data.
- various aspects of the method provided by this application can also be implemented in the form of a program product, which includes program code.
- the program product When the program product is run on a computer device, the program code is used to The computer device is caused to execute the steps in the method described above in this specification according to various exemplary embodiments of the present application.
- the computer device may execute the video image noise assessment method described in the embodiments of the present application.
- the program product may be implemented in any combination of one or more readable media.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
本申请实施例提供了一种视频图像噪声评估方法及装置，该方法包括：对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据；对所述视频图像进行区域划分得到前景区域和背景区域；根据所述噪声数据确定所述视频图像的全图噪声信息，以及所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪声信息，本方案可以实现噪声点的精确定位，能够精准评估视频图像的噪声水平，获得更有目的性也更加符合主观认知的噪声水平估计，为后续的定向降噪处理提供良好支撑。
Description
本申请要求在2022年06月10日提交中国专利局，申请号为202210658211.9的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
本申请实施例涉及噪声识别技术领域，尤其涉及一种视频图像噪声评估方法及装置。
随着直播视频业务的持续发展和应用普及，直播视频已经成为普通人生活社交中非常重要的一部分，与此同时观众对于直播视频本身质量和画质也产生了更高的要求。而视频画面中是否有较多的噪声点是显著影响受众和主播视觉观感的一个衡量因素，而噪声作为一种画面中出现的异常信号，成因复杂同时视觉表现各异，所以精准地刻画视频画面中的噪声也是提升直播质量非常关键的一个环节，其可以有效地评估视频噪声水平并且指导降噪工作。
相关技术中，存在从信号滤波的角度出发进行噪声估计的方案，该种方式中噪声的估计是粗略的，且并不稳定，在实际的应用中效果有限同时耗时过长；针对采用深度学习方法进行噪声确定的方案，其确定的噪声结果同样细致程度不够，其确定的噪声无法在后续实现局部降噪处理，且使用的深度学习模型如果要取得比较满意的结果，通常要求大量精细标注过的数据进行模型训练，充足且完整标注的大量视频噪声数据通常不存在且需要大量精力完成高质量标注，因此实现过程相对复杂。
发明内容
本申请实施例提供了一种视频图像噪声评估方法及装置，可以实现噪声点的精确定位，能够精准评估视频图像的噪声水平，为后续的定向降噪处理提供良好支撑。
第一方面，本申请实施例提供了一种视频图像噪声评估方法，该方法包括：
对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据；
对所述视频图像进行区域划分得到前景区域和背景区域；
根据所述噪声数据确定所述视频图像的全图噪声信息，以及所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪声信息。
第二方面，本申请实施例还提供了一种视频图像噪声评估装置，包括：
像素噪声确定模块，配置为对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据；
图像区域划分模块，配置为对所述视频图像进行区域划分得到前景区域和背景区域；
噪声信息确定模块，配置为根据所述噪声数据确定所述视频图像的全图噪声信息，以及所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪声信息。
第三方面，本申请实施例还提供了一种视频图像噪声评估设备，该设备包括：
一个或多个处理器；
存储装置，用于存储一个或多个程序，
当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现本申请实施例所述的视频图像噪声评估方法。
第四方面，本申请实施例还提供了一种存储计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行本申请实施例所述的视频图像噪声评估方法。
第五方面，本申请实施例还提供了一种计算机程序产品，该计算机程序产品包括计算机程序，该计算机程序存储在计算机可读存储介质中，设备的至少一个处理器从计算机可读存储介质读取并执行计算机程序，使得设备执行本申请实施例所述的视频图像噪声评估方法。
本申请实施例中，通过对视频图像进行噪声估计得到视频图像中每个像素点的噪声数据，对视频图像进行区域划分得到前景区域和背景区域，根据噪声数据确定视频图像的全图噪声信息，以及前景区域对应的前景噪声信息和背景区域对应的背景噪声信息，该种噪声评估方法更加精细，细化到像素点的噪声情况，同时采用划分区域并对各个区域进行单独的噪声信息的评估，可以获得更有目的性也更加符合主观认知的噪声水平估计，为后续的定向降噪处理提供良好支撑。
图1为本申请实施例提供的一种视频图像噪声评估方法的流程图；
图2为本申请实施例提供的一种噪声图可视化展示的示意图；
图3为本申请实施例提供的一种将视频图像噪声覆盖至原始视频图像的示意图；
图4为本申请实施例提供的一种包含对输入图像进行变换调整的视频图像噪声评估方法的流程图；
图5为本申请实施例提供的一种包含对视频图像进行边缘处理的视频图像噪声评估方法的流程图；
图6为本申请实施例提供的一种对视频图像进行区域划分的视频图像噪声评估方法的流程图；
图7为本申请实施例提供的一种将视频图像划分为前景区域和背景区域的示意图；
图8为本申请实施例提供的一种确定全图噪声信息的视频图像噪声评估方法的流程图；
图9为本申请实施例提供的一种确定前景区域和背景区域噪声信息的视频图像噪声评估方法的流程图；
图10为本申请实施例提供的一种视频图像噪声评估装置的结构框图；
图11为本申请实施例提供的一种视频图像噪声评估设备的结构示意图。
下面结合附图和实施例对本申请实施例作进一步的详细说明。可以理解的是，此处所描述的具体实施例仅仅用于解释本申请实施例，而非对本申请实施例的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与本申请实施例相关的部分而非全部结构。
本申请的说明书和权利要求书中的术语“第一”、“第二”等是用于区别类似的对象，而不用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施，且“第一”、“第二”等所区分的对象通常为一类，并不限定对象的个数，例如第一对象可以是一个，也可以是多个。此外，说明书
以及权利要求中“和/或”表示所连接对象的至少其中之一，字符“/”，一般表示前后关联对象是一种“或”的关系。
图1为本申请实施例提供的一种视频图像噪声评估方法的流程图，可用于对视频图像或者单一图像进行噪声评估，确定图像的噪声情况，该方法可以由计算设备如服务器、智能终端、笔记本、平板电脑等来执行，以服务器为执行设备为例，具体可以是视频后端处理的Linux服务器，具体包括如下步骤：
步骤S101、对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据。
在一个实施例中，通过集成的算法模块进行视频图像噪声评估，对输入的视频图像进行噪声估计得到视频图像中每个像素点的噪声数据。其中，该视频图像可以是直播视频的图像画面，或者输入的单独的静态图像等。可选的，该噪声数据可以为像素点对应的噪声值，即针对视频图像中的每个像素点得到该像素点的具体的噪声值。示例性的，噪声值的取值范围为0至1，其中，取值越大表征噪声越明显。
可选的，对视频图像进行噪声估计过程为：通过训练完成的多层深度学习神经网络对视频图像进行噪声估计，得到视频图像中每个像素点的噪声数据。其中，该多层深度学习神经网络包括堆叠设置的多个残差网络模块，针对输入的视频图像，通过该多层深度学习神经网络输出与输入视频图像一致大小的噪声图。可选的，该噪声图可以以矩阵的形式表征并存储，矩阵中的每个元素对应视频图像中的一个像素点，具体的元素的值为像素点的噪声值。
在一个实施例中，通过多层深度学习神经网络对视频图像进行噪声估计，输出噪声图后，对该噪声图进行可视化显示。示例性的，如图2所示，图2为本申请实施例提供的一种噪声图可视化展示的示意图，图中亮点区域即为视频图像中的噪声部分，越集中且亮度越高的点对应的视频图像的噪声越明显。可选的，在进行噪声图可视化展示的过程中，可将噪声点叠加至原图进行显示。示例性的，如图3所示，图3为本申请实施例提供的一种将视频图像噪声覆盖至原始视频图像的示意图，以此可以更直观的进行视频图像噪声的观察。
步骤S102、对所述视频图像进行区域划分得到前景区域和背景区域。
在一个实施例中，进行视频图像的噪声估计时，对视频图像进行区域划分得到前景区域和背景区域。其中区域划分的目的在于，由于晕影效应存使得产生的噪声不均衡，且用户人眼集中观看的区域有限，通常集中在中心区域，故
进一步的视频图像进行区域划分得到前景区域和背景区域，其中，前景区域为划分的用户人眼集中观看的区域，背景区域为划分的用户人眼非集中观看的区域。并在后续噪声估计时，基于划分的区域进行分区的噪声统计。
可选的，前景区域可以是以视频图像画面为中心，框选的固定尺寸大小的矩形、圆形等图形所围成的区域。相应的，所围成区域以外的视频图像的区域为背景区域。
步骤S103、根据所述噪声数据确定所述视频图像的全图噪声信息，以及所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪声信息。
在一个实施例中，确定出细粒度的像素点的噪声数据后，分别确定划分的前景区域和背景区域前景噪声信息和背景区域，以及全图噪声信息，通过对该三种噪声信息的统计以真实反映每张输入的视频图像的噪声水平。其中，全图噪声信息表征视频图像的整图的噪声情况，前景噪声信息表征人眼关注的区域的噪声情况，背景噪声信息表征非人眼关注的区域的噪声情况。即基于确定出的噪声数据进行分区域的噪声情况的评估，以反映真实的符合人眼主观感受的噪声情况。示例性的，针对相同噪声值的情况，噪声出现在前景区域相对于噪声出现在背景区域的情况，视频图像质量越差，噪声情况越严重。
在一个实施例中，在确定出视频图像的全图噪声信息、前景噪声信息和背景噪声信息后，分别输出包含该三个变量的视频图像的信息，如以直观的曲线图的形式输出，以进行分区域的噪声水平的展示。
由上述方案可知，通过对视频图像进行噪声估计得到视频图像中每个像素点的噪声数据，对视频图像进行区域划分得到前景区域和背景区域，根据噪声数据确定视频图像的全图噪声信息，以及前景区域对应的前景噪声信息和背景区域对应的背景噪声信息，该种噪声评估方法更加精细，细化到像素点的噪声情况，同时采用划分区域并对各个区域进行单独的噪声信息的评估，可以获得更有目的性也更加符合主观认知的噪声水平估计，为后续的定向降噪处理提供良好支撑。
图4为本申请实施例提供的一种包含对输入图像进行变换调整的视频图像噪声评估方法的流程图，如图4所示，具体包括：
步骤S201、在确定出输入的视频图像的分辨率小于预设分辨率大小的情况下，对所述视频图像进行边缘填补得到所述预设分辨率大小的视频图像，对调整后的视频图像进行图像像素值的归一化处理。
在一个实施例中，进行视频图像的噪声评估时，针对固定大小尺寸的视频图像进行噪声评估处理，以提高处理效率。在进行视频图像的评估前，确定该视频图像的分辨率和设置的预设分辨率大小是否相同，如果不同则相应的进行调整。示例性的，设置的预设分辨率大小1280x720(720P)。通过大量对比实验得出，将较高分辨率的图像(如720P图像)等比例缩放到较低分辨率(如540P或360P)图像时，其噪声水平的估计在缩放过程中会扭曲和损失，无法实现一个等比例系数的图像噪声的无损缩放，故此时，在确定出输入的视频图像的分辨率小于预设分辨率大小的情况下，对视频图像进行边缘填补得到预设分辨率大小的视频图像。示例性的，如对输入的视频图像进行黑色边缘区域的填充以得到固定尺寸大小的视频图像。同时将调整后的视频图像进行图像像素值的归一化处理，将原有的像素值的取值区间由[0，255]调整为[0，1]的区间以便于进行像素点的噪声数据的确定，提高运算效率。
在一个实施例中，还包括对输入图像的录制模式进行识别，示例性的，录制模式包括水平录制模式和垂直录制模式，假定设置的进行视频图像噪声评估的输入的视频图像为垂直模式，则检测到当前视频图像为非垂直模式，则将其调整为垂直模式。示例性的，假定当前的视频图像的录制模式为水平模式，则将视频图像旋转90°以调整为垂直模式。
在另一个实施例中，针对输入的视频图像的分辨率高于设置的预设分辨率大小的情况，可以是采用对输入的视频图像进行裁剪或等比例缩小的方式以得到和设置的固定尺寸大小的视频图像。
步骤S202、对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据。
步骤S203、对所述视频图像进行区域划分得到前景区域和背景区域。
步骤S204、根据所述噪声数据确定所述视频图像的全图噪声信息，以及所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪声信息。
由上述可知，在确定出输入的视频图像的分辨率小于预设分辨率大小的情况下，对视频图像进行边缘填补得到预设分辨率大小的视频图像，对调整后的视频图像进行图像像素值的归一化处理，使得进行噪声评估的视频图像更利于高效的进行噪声数据的确定，提高了数据处理效率，简化了算法模块的计算处理过程。
图5为本申请实施例提供的一种包含对视频图像进行边缘处理的视频图像
噪声评估方法的流程图，如图5所示，具体包括：
步骤S301、在确定出输入的视频图像的分辨率小于预设分辨率大小的情况下，对所述视频图像进行边缘填补得到所述预设分辨率大小的视频图像，对调整后的视频图像进行图像像素值的归一化处理。
步骤S302、对分辨率调整以及归一化处理后的视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据。
步骤S303、对调整后的视频图像进行尺寸大小以及图像像素值的恢复，对恢复后的视频图像进行边缘检测得到高频边缘信息，对所述高频边缘信息进行剔除。
在一个实施例中，进行调整后的视频图像进行尺寸大小的恢复的过程可以是，将边缘填充时填充的区域进行删除，即恢复原有的图像大小尺寸，符合真实的视频录制或图像拍摄的大小，优化噪声评估的直观显示效果。同时，在进行区域划分前，对恢复后的视频图像进行边缘检测得到高频边缘信息，对高频边缘信息进行剔除。可选的，对恢复后的视频图像进行边缘检测的方式可以是利用边缘检测Canny算法将得到的噪声数据中潜在存在的高频边缘信息与噪声信号进行分离，即对高频的边缘进行剔除，以避免后续进行噪声信息的确定时对噪声值的确定产生影响。
步骤S304、对进行恢复处理以及高频边缘剔除后的视频图像进行区域划分得到前景区域和背景区域。
步骤S305、根据所述噪声数据确定所述视频图像的全图噪声信息，以及所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪声信息。
由上述可知，通过对调整后的视频图像进行恢复，以便于视频图像的直观的显示，同时进行高频边缘信息进行剔除，避免了视频图像中锐利的边缘对噪声信息的确定产生影响。
图6为本申请实施例提供的一种对视频图像进行区域划分的视频图像噪声评估方法的流程图，如图6所示，具体包括：
步骤S401、对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据。
步骤S402、以所述视频图像的中心为原点，构造内接椭圆，将所述内接椭圆所在的图像区域确定为前景区域，将所述内接椭圆以外的图像区域确定为背景区域。
在一个实施例中，进行视频图像区域的划分时，以视频图像的中心为原点，构造内接椭圆，其中，内接椭圆的面积大小根据视频图像的尺寸以及预设的调节参数确定。示例性的，构造的椭圆可表示为：
其中，参数H和W分别为视频图像的高和宽，ratio为预设的调节参数。
在一个实施例中，通过对预设的调节参数的大小进行设置以满足不同的观察需要和具体的不同业务场景，示例性的，将其设置为0.8。如图7所示，图7为本申请实施例提供的一种将视频图像划分为前景区域和背景区域的示意图，其中，4021为前景区域，4022为背景区域。
步骤S403、根据所述噪声数据确定所述视频图像的全图噪声信息，以及所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪声信息。
由上述可知，通过以视频图像的中心为原点，构造内接椭圆，将内接椭圆所在的图像区域确定为前景区域，将内接椭圆以外的图像区域确定为背景区域，进行符合人眼主观视觉观察的区域划分，以进行相应的不同区域的噪声信息的标定，获得更有目的性也更加符合主观认知的噪声水平估计，为后续的定向降噪处理提供良好支撑。
图8为本申请实施例提供的一种确定全图噪声信息的视频图像噪声评估方法的流程图，如图8所示，具体包括：
步骤S501、对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据。
步骤S502、对所述视频图像进行区域划分得到前景区域和背景区域。
步骤S503、获取预先基于图像数据集计算得到的平均衡量阈值，对所述噪声数据中大于所述平均衡量阈值的像素点的噪声值进行筛选，对筛选的像素点的噪声值进行均值计算得到视频图像的全图噪声信息。
在一个实施例中，首先基于图像数据集进行分析计算得到平均衡量阈值。该平均衡量阈值示例性的可以是通过统计分析计算得到的值，即像素点的噪声值大于该平均衡量阈值时，为相对明显的噪声。对于输入的视频图像中的每个像素点确定出的噪声值中，对大于该平均衡量阈值的像素点的噪声值进行筛选，
即筛选出相对明显的噪声值的点，对筛选的像素点的噪声值进行均值计算得到视频图像的全图噪声信息。
可选的，由于人眼对于处于亮度中间的值，比其他过亮或者过暗的区域更为敏感，且视觉敏感度随亮度增加呈现非均匀的增长，在计算全图噪声信息时，还可以是：确定每个筛选的像素点的亮度值，基于所述亮度值确定对应噪声值的权重；对筛选的像素点的噪声值分别乘以对应的权重后，进行均值计算得到所述视频图像的全图噪声信息。具体的根据像素点的亮度计算对应噪声值的权重的方式可以是如下公式：
fweighted＝a(x-b)2+c
fweighted＝a(x-b)2+c
其中，fweigted为计算得到的权重值，x为每个像素点的亮度值，a、b和c为具体的函数方程的参数，示例性的，a的取值区间可以是[1.5，3]，b的取值范围可以是[30，50]，c的取值范围为[1.5，3]。
其中，该图像数据集可以是包含该视频图像的整个视频的图像数据集。示例性的，以噪声值的取值范围为[0，1]为例，该平均衡量阈值通过计算确定为0.76。可选的，
步骤S504、根据所述噪声数据确定所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪声信息。
由上述可知，在确定全图噪声信息时，利用各个像素点的噪声值进行计算，首先进行明显噪声值的筛选，对于筛选出的噪声值采用加权平均的方式进行计算得到全图噪声信息，其中，权重值依据像素点的亮度值计算得到，使得最终计算的噪声信息更加符合人眼主观感受，噪声评估效果更佳。
图9为本申请实施例提供的一种确定前景区域和背景区域噪声信息的视频图像噪声评估方法的流程图，如图9所示，具体包括：
步骤S601、对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据。
步骤S602、对所述视频图像进行区域划分得到前景区域和背景区域。
步骤S603、根据所述噪声数据确定所述视频图像的全图噪声信息。
步骤S604、获取预先基于图像数据集计算得到的平均衡量阈值，对所述前景区域和所述背景区域的噪声数据中大于所述平均衡量阈值的像素点的噪声值进行筛选得到前景噪声值和背景噪声值。
在一个实施例中，首先基于图像数据集进行分析计算得到平均衡量阈值。该平均衡量阈值示例性的可以是通过统计分析计算得到的值，即像素点的噪声值大于该平均衡量阈值时，为相对明显的噪声。对于输入的视频图像中的每个像素点确定出的噪声值中，对大于该平均衡量阈值的像素点的噪声值进行筛选，即筛选出相对明显的噪声值的点，分别得到前景区域和背景区域对应的前景噪声值和背景噪声值。
步骤S605、根据所述前景噪声值和所述前景区域的区域大小计算得到前景噪声值占比，以及根据所述背景噪声值和所述背景区域的区域大小计算得到背景噪声值占比。
在一个实施例中，确定前景噪声信息和背景噪声信息的方式可以是确定噪声值占比。具体的，针对前景区域而言，计算该前景区域中筛选出的噪声值之和，除以前景区域的面积大小以得到前景噪声值占比。同理，针对背景区域而言，计算该背景区域中筛选出的噪声值之和，除以背景区域的面积大小以得到背景噪声值占比，即通过噪声值占比表征划分的对应区域的噪声情况。
可选的，以前景区域中筛选出的像素点的噪声值分别为noise1至noisei，再分别计算每个噪声值对应的权重值，该权重值基于对应的像素点的亮度值计算得到，具体计算方式参见步骤S503的解释部分，此处不再赘述。假设噪声值noise1至noisei分别对应的权重值记为weigt1至weigti，筛选出的噪声值的加权求和的计算公式如下：
weightedSum＝noise1*weight1+…noisei*weighti
weightedSum＝noise1*weight1+…noisei*weighti
相应的，针对前景区域计算的前景噪声信息以前景噪声值占比的方式表征时，计算公式如下：
其中，针对背景区域的背景噪声值占比的计算方式同理，此处不再赘述。
由上述可知，在确定区域的噪声信息时，利用各个像素点的噪声值进行计算，首先进行明显噪声值的筛选，对于筛选出的噪声值采用加权平均的方式进行计算得到区域噪声信息，以表征不同人眼主观感受下的噪声评估，其中，权重值依据像素点的亮度值计算得到，使得最终计算的噪声信息更加符合人眼主观感受，噪声评估效果更佳。
图10为本申请实施例提供的一种视频图像噪声评估装置的结构框图，该装
置用于执行上述实施例提供的视频图像噪声评估方法，具备执行方法相应的功能模块和有益效果。如图10所示，该装置具体包括：像素噪声确定模块101、图像区域划分模块102和噪声信息确定模块103，其中，
像素噪声确定模块101，配置为对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据；
图像区域划分模块102，配置为对所述视频图像进行区域划分得到前景区域和背景区域；
噪声信息确定模块103，配置为根据所述噪声数据确定所述视频图像的全图噪声信息，以及所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪声信息。
由上述方案可知，通过对视频图像进行噪声估计得到视频图像中每个像素点的噪声数据，对视频图像进行区域划分得到前景区域和背景区域，根据噪声数据确定视频图像的全图噪声信息，以及前景区域对应的前景噪声信息和背景区域对应的背景噪声信息，该种噪声评估方法更加精细，细化到像素点的噪声情况，同时采用划分区域并对各个区域进行单独的噪声信息的评估，可以获得更有目的性也更加符合主观认知的噪声水平估计，为后续的定向降噪处理提供良好支撑。
在一个可能的实施例中，所述像素噪声确定模块，配置为：
通过多层深度学习神经网络对视频图像进行噪声估计，得到所述视频图像中每个像素点的噪声数据，所述多层深度学习神经网络包括堆叠设置的多个残差网络模块；
基于所述噪声数据生成所述视频图像对应的可视化噪声图。
在一个可能的实施例中，该装置还包括图像处理模块，配置为：
在所述对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据之前，在确定出输入的视频图像的分辨率小于预设分辨率大小的情况下，对所述视频图像进行边缘填补得到所述预设分辨率大小的视频图像；
对调整后的视频图像进行图像像素值的归一化处理。
在一个可能的实施例中，所述图像处理模块，配置为：
在所述对所述视频图像进行区域划分得到前景区域和背景区域之前，对恢复后的视频图像进行边缘检测得到高频边缘信息，对所述高频边缘信息进行剔除。
在一个可能的实施例中，所述图像区域划分模块，配置为：
以所述视频图像的中心为原点，构造内接椭圆，所述内接椭圆的面积大小根据所述视频图像的尺寸以及预设的调节参数确定；
将所述内接椭圆所在的图像区域确定为前景区域，将所述内接椭圆以外的图像区域确定为背景区域。
在一个可能的实施例中，所述噪声信息确定模块，配置为：
获取预先基于图像数据集计算得到的平均衡量阈值，对所述噪声数据中大于所述平均衡量阈值的像素点的噪声值进行筛选；
对筛选的像素点的噪声值进行均值计算得到所述视频图像的全图噪声信息。
在一个可能的实施例中，所述噪声信息确定模块，配置为：
确定每个筛选的像素点的亮度值，基于所述亮度值确定对应噪声值的权重；
对筛选的像素点的噪声值分别乘以对应的权重后，进行均值计算得到所述视频图像的全图噪声信息。
在一个可能的实施例中，所述噪声信息确定模块，配置为：
获取预先基于图像数据集计算得到的平均衡量阈值，对所述前景区域和所述背景区域的噪声数据中大于所述平均衡量阈值的像素点的噪声值进行筛选得到前景噪声值和背景噪声值；
根据所述前景噪声值和所述前景区域的区域大小计算得到前景噪声值占比，以及根据所述背景噪声值和所述背景区域的区域大小计算得到背景噪声值占比。
在一个可能的实施例中，所述噪声信息确定模块，配置为：
将每个所述前景噪声值和对应的权重的乘积之和，除以所述前景区域的区域大小，得到前景噪声值占比；
将每个所述背景噪声值和对应的权重的乘积之和，除以所述背景区域的区域大小，得到背景噪声值占比，其中，所述前景噪声值和所述背景噪声值的权重基于对应的像素点的亮度值计算得到。
图11为本申请实施例提供的一种视频图像噪声评估设备的结构示意图，如图11所示，该设备包括处理器201、存储器202、输入装置203和输出装置204；设备中处理器201的数量可以是一个或多个，图11中以一个处理器201为例；设备中的处理器201、存储器202、输入装置203和输出装置204可以通过总线
或其他方式连接，图11中以通过总线连接为例。存储器202作为一种计算机可读存储介质，可用于存储软件程序、计算机可执行程序以及模块，如本申请实施例中的视频图像噪声评估方法对应的程序指令/模块。处理器201通过运行存储在存储器202中的软件程序、指令以及模块，从而执行设备的各种功能应用以及数据处理，即实现上述的视频图像噪声评估方法。输入装置203可用于接收输入的数字或字符信息，以及产生与设备的用户设置以及功能控制有关的键信号输入。输出装置204可包括显示屏等显示设备。
本申请实施例还提供一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行一种上述实施例描述的视频图像噪声评估方法，其中，包括：
对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据；
对所述视频图像进行区域划分得到前景区域和背景区域；
根据所述噪声数据确定所述视频图像的全图噪声信息，以及所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪声信息。
值得注意的是，上述视频图像噪声评估装置的实施例中，所包括的各个单元和模块只是按照功能逻辑进行划分的，但并不局限于上述的划分，只要能够实现相应的功能即可；另外，各功能单元的具体名称也只是为了便于相互区分，并不用于限制本申请实施例的保护范围。
在一些可能的实施方式中，本申请提供的方法的各个方面还可以实现为一种程序产品的形式，其包括程序代码，当所述程序产品在计算机设备上运行时，所述程序代码用于使所述计算机设备执行本说明书上述描述的根据本申请各种示例性实施方式的方法中的步骤，例如，所述计算机设备可以执行本申请实施例所记载的视频图像噪声评估方法。所述程序产品可以采用一个或多个可读介质的任意组合实现。
Claims (13)
- 视频图像噪声评估方法，其中，包括：对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据；对所述视频图像进行区域划分得到前景区域和背景区域；根据所述噪声数据确定所述视频图像的全图噪声信息，以及所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪声信息。
- 根据权利要求1所述的视频图像噪声评估方法，其中，所述对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据，包括：通过多层深度学习神经网络对视频图像进行噪声估计，得到所述视频图像中每个像素点的噪声数据，所述多层深度学习神经网络包括堆叠设置的多个残差网络模块；基于所述噪声数据生成所述视频图像对应的可视化噪声图。
- 根据权利要求1或2所述的视频图像噪声评估方法，其中，在所述对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据之前，还包括：在确定出输入的视频图像的分辨率小于预设分辨率大小的情况下，对所述视频图像进行边缘填补得到所述预设分辨率大小的视频图像；对调整后的视频图像进行图像像素值的归一化处理。
- 根据权利要求3所述的视频图像噪声评估方法，其中，在所述对所述视频图像进行区域划分得到前景区域和背景区域之前，还包括：对调整后的视频图像进行尺寸大小以及图像像素值的恢复；对恢复后的视频图像进行边缘检测得到高频边缘信息，对所述高频边缘信息进行剔除。
- 根据权利要求1-4中任一项所述的视频图像噪声评估方法，其中，所述对所述视频图像进行区域划分得到前景区域和背景区域，包括：以所述视频图像的中心为原点，构造内接椭圆，所述内接椭圆的面积大小根据所述视频图像的尺寸以及预设的调节参数确定；将所述内接椭圆所在的图像区域确定为前景区域，将所述内接椭圆以外的图像区域确定为背景区域。
- 根据权利要求1-5中任一项所述的视频图像噪声评估方法，其中，所述根据所述噪声数据确定所述视频图像的全图噪声信息，包括：获取预先基于图像数据集计算得到的平均衡量阈值，对所述噪声数据中大于所述平均衡量阈值的像素点的噪声值进行筛选；对筛选的像素点的噪声值进行均值计算得到所述视频图像的全图噪声信息。
- 根据权利要求6所述的视频图像噪声评估方法，其中，所述对筛选的像素点的噪声值进行均值计算得到所述视频图像的全图噪声信息，包括：确定每个筛选的像素点的亮度值，基于所述亮度值确定对应噪声值的权重；对筛选的像素点的噪声值分别乘以对应的权重后，进行均值计算得到所述视频图像的全图噪声信息。
- 根据权利要求1-7中任一项所述的视频图像噪声评估方法，其中，根据所述噪声数据确定所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪声信息，包括：获取预先基于图像数据集计算得到的平均衡量阈值，对所述前景区域和所述背景区域的噪声数据中大于所述平均衡量阈值的像素点的噪声值进行筛选得到前景噪声值和背景噪声值；根据所述前景噪声值和所述前景区域的区域大小计算得到前景噪声值占比，以及根据所述背景噪声值和所述背景区域的区域大小计算得到背景噪声值占比。
- 根据权利要求8所述的视频图像噪声评估方法，其中，所述根据所述前景噪声值和所述前景区域的区域大小计算得到前景噪声值占比，以及根据所述背景噪声值和所述背景区域的区域大小计算得到背景噪声值占比，包括：将每个所述前景噪声值和对应的权重的乘积之和，除以所述前景区域的区域大小，得到前景噪声值占比；将每个所述背景噪声值和对应的权重的乘积之和，除以所述背景区域的区域大小，得到背景噪声值占比，其中，所述前景噪声值和所述背景噪声值的权重基于对应的像素点的亮度值计算得到。
- 视频图像噪声评估装置，其中，包括：像素噪声确定模块，配置为对视频图像进行噪声估计得到所述视频图像中每个像素点的噪声数据；图像区域划分模块，配置为对所述视频图像进行区域划分得到前景区域和背景区域；噪声信息确定模块，配置为根据所述噪声数据确定所述视频图像的全图噪声信息，以及所述前景区域对应的前景噪声信息和所述背景区域对应的背景噪 声信息。
- 一种视频图像噪声评估设备，所述设备包括：一个或多个处理器；存储装置，用于存储一个或多个程序，当所述一个或多个程序被所述一个或多个处理器执行，使得所述一个或多个处理器实现权利要求1-9中任一项所述的视频图像噪声评估方法。
- 一种存储计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行权利要求1-9中任一项所述的视频图像噪声评估方法。
- 一种计算机程序产品，包括计算机程序，其中，所述计算机程序被处理器执行时实现权利要求1-9中任一项所述的视频图像噪声评估方法。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210658211.9 | 2022-06-10 | ||
| CN202210658211.9A CN114881889A (zh) | 2022-06-10 | 2022-06-10 | 视频图像噪声评估方法及装置 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2023236757A1 true WO2023236757A1 (zh) | 2023-12-14 |
Family
ID=82680715
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2023/095180 WO2023236757A1 (zh) | 2022-06-10 | 2023-05-19 | 视频图像噪声评估方法及装置 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114881889A (zh) |
| WO (1) | WO2023236757A1 (zh) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114881889A (zh) * | 2022-06-10 | 2022-08-09 | 百果园技术(新加坡)有限公司 | 视频图像噪声评估方法及装置 |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530248A (zh) * | 2016-10-28 | 2017-03-22 | 中国南方电网有限责任公司 | 一种变电站场景视频噪声的智能检测方法 |
| CN111583160A (zh) * | 2020-06-03 | 2020-08-25 | 浙江大华技术股份有限公司 | 一种视频画面的噪声评估方法及装置 |
| CN114881889A (zh) * | 2022-06-10 | 2022-08-09 | 百果园技术(新加坡)有限公司 | 视频图像噪声评估方法及装置 |
-
2022
- 2022-06-10 CN CN202210658211.9A patent/CN114881889A/zh active Pending
-
2023
- 2023-05-19 WO PCT/CN2023/095180 patent/WO2023236757A1/zh unknown
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530248A (zh) * | 2016-10-28 | 2017-03-22 | 中国南方电网有限责任公司 | 一种变电站场景视频噪声的智能检测方法 |
| CN111583160A (zh) * | 2020-06-03 | 2020-08-25 | 浙江大华技术股份有限公司 | 一种视频画面的噪声评估方法及装置 |
| CN114881889A (zh) * | 2022-06-10 | 2022-08-09 | 百果园技术(新加坡)有限公司 | 视频图像噪声评估方法及装置 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114881889A (zh) | 2022-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8698910B2 (en) | Apparatus, camera, method, and computer-readable storage medium for generating advice for capturing an image | |
| CN104272346B (zh) | 用于细节增强和降噪的图像处理方法 | |
| EP3770747A1 (en) | Method for displaying a picture and method for rotating a displayed picture | |
| US20130027583A1 (en) | Photographing apparatus, method and program | |
| US8908990B2 (en) | Image processing apparatus, image processing method, and computer readable medium for correcting a luminance value of a pixel for reducing image fog | |
| US20100201851A1 (en) | Digital camera incorporating a sharpness predictor | |
| US8837856B2 (en) | Image processing apparatus, image processing method, and computer readable medium | |
| US8437542B2 (en) | Image processing apparatus, method, and program | |
| CN102402918B (zh) | 改善画面画质的方法及lcd显示器 | |
| WO2012025042A1 (zh) | 视频画面显示方法及装置 | |
| WO2019041842A1 (zh) | 一种图像处理方法及装置、存储介质和计算机设备 | |
| WO2023236757A1 (zh) | 视频图像噪声评估方法及装置 | |
| US11074742B2 (en) | Image processing apparatus, image processing method, and storage medium | |
| CN110443783B (zh) | 一种图像质量评估方法及装置 | |
| WO2022156129A1 (zh) | 图像处理方法、图像处理装置及计算机设备 | |
| US8896692B2 (en) | Apparatus, system, and method of image processing, and recording medium storing image processing control program | |
| CN112767392A (zh) | 图像清晰度确定方法、装置、设备及存储介质 | |
| CN112150368A (zh) | 图像处理方法、装置、电子设备及计算机可读存储介质 | |
| US8958637B2 (en) | Image processing apparatus, image processing method, and computer readable medium | |
| CN113990263A (zh) | 显示屏的背光处理方法、装置、存储介质及电子设备 | |
| US20190304072A1 (en) | Method and apparatus for converting low dynamic range video to high dynamic range video | |
| US11410285B2 (en) | Method and apparatus for image processing | |
| US20190166299A1 (en) | Image processing apparatus, control method thereof, and non-transitory computer-readable storage medium | |
| CN111277724B (zh) | 异常运行应用的检测方法、装置、电子设备及存储介质 | |
| CN110782392B (zh) | 图像处理方法、装置、电子设备及存储介质 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23818922 Country of ref document: EP Kind code of ref document: A1 |