WO2017190415A1 - 一种图像优化方法、装置及终端 - Google Patents

一种图像优化方法、装置及终端 Download PDF

Info

Publication number
WO2017190415A1
WO2017190415A1 PCT/CN2016/088609 CN2016088609W WO2017190415A1 WO 2017190415 A1 WO2017190415 A1 WO 2017190415A1 CN 2016088609 W CN2016088609 W CN 2016088609W WO 2017190415 A1 WO2017190415 A1 WO 2017190415A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
depth
pixel
matrix
optimized
Prior art date
Application number
PCT/CN2016/088609
Other languages
English (en)
French (fr)
Inventor
胡文迪
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Priority to US16/097,282 priority Critical patent/US20190139198A1/en
Priority to EP16900937.0A priority patent/EP3438921A4/en
Publication of WO2017190415A1 publication Critical patent/WO2017190415A1/zh

Links

Images

Classifications

    • G06T5/70
    • G06T5/73
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/136Segmentation; Edge detection involving thresholding
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/194Segmentation; Edge detection involving foreground-background segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/50Depth or shape recovery
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70Circuitry for compensating brightness variation in the scene
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70Circuitry for compensating brightness variation in the scene
    • H04N23/72Combination of two or more compensation controls
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70Circuitry for compensating brightness variation in the scene
    • H04N23/76Circuitry for compensating brightness variation in the scene by influencing the image signals
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof
    • H04N23/81Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof
    • H04N23/84Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/88Camera processing pipelines; Components thereof for processing colour signals for colour balance, e.g. white-balance circuits or colour temperature control
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment
    • H04N5/2224Studio circuitry; Studio devices; Studio equipment related to virtual studio applications
    • H04N5/2226Determination of depth image, e.g. for foreground/background separation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/64Circuits for processing colour signals
    • H04N9/73Colour balance circuits, e.g. white balance circuits or colour temperature control
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10004Still image; Photographic image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10028Range image; Depth image; 3D point clouds
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20021Dividing image into blocks, subimages or windows

Definitions

  • This document relates to, but is not limited to, the field of image processing, and in particular, to an image optimization method, apparatus and terminal.
  • the related technology processing method is mainly based on the main scene of the picture. For example, if the main body is an indoor scene, the optimization effect of the external scenery of the window is sacrificed, and the outdoor scene will be Overexposure and color cast.
  • the technician proposes to reduce the overexposure and color cast of the outdoor scene by balancing the outdoor scene and the indoor scene, but adjusting the weight of the indoor and outdoor scene will inevitably lead to partial optimization of the subject scene, indoors and outdoors. It is difficult to achieve a better balance of optimization results.
  • This paper provides an image optimization method, device and terminal, which can solve the problem of sacrificing background optimization effect in the image optimization method in the related art.
  • This article provides an image optimization method that includes:
  • the image to be optimized is optimized according to the picture depth information.
  • the acquiring the depth of field information of the image to be optimized includes:
  • the image to be optimized is measured by a dual camera algorithm, a laser focusing method, or a software algorithm to obtain the depth of field information of the screen.
  • the optimizing the image to be optimized according to the depth of field information includes:
  • Different automatic white balance and/or automatic exposure control are used for indoor and outdoor areas, respectively.
  • the dividing the image to be optimized into an indoor area and an outdoor area according to the picture depth information includes:
  • the difference between the depth values of the pixels is less than the threshold; the ratio of the average depth of field values of each target area to the adjacent target area and the ratio of the white balance/exposure average are calculated, when the average of a target area and its adjacent target area
  • the ratio of the depth of field value and the ratio of the white balance/exposure average are both greater than the corresponding threshold, the target area and its adjacent target area are respectively determined as the indoor area and the outdoor area.
  • the optimizing the image to be optimized according to the depth of field information includes:
  • the calculating a denoising matrix and a sharpening matrix of each pixel in the image to be optimized according to the depth information of the screen includes:
  • the noise matrix and the sharpening matrix calculate a denoising matrix and a sharpening matrix of each pixel.
  • the depth of field values of each pixel point are normalized, and the matrix weighting coefficients of each pixel point are obtained:
  • the depth of field value, ⁇ a is the matrix weighting coefficient of pixel point a.
  • an image optimization apparatus comprising:
  • an optimization module configured to optimize the image to be optimized according to the picture depth information.
  • the acquiring, by the acquiring module, the screen depth information of the image to be optimized includes:
  • the image to be optimized is measured by a dual camera algorithm, a laser focusing method, or a software algorithm to obtain the depth of field information of the screen.
  • the optimizing module optimizes the image to be optimized according to the depth of field information of the screen, including:
  • Different automatic white balance and/or automatic exposure control are used for indoor and outdoor areas, respectively.
  • the optimization module according to the picture depth information, to divide the image to be optimized into an indoor area and an outdoor area, includes:
  • the difference between the depth values of the adjacent pixels is less than the threshold; calculating the ratio of the average depth value of each target region to the adjacent target region and the ratio of the white balance/exposure average, when a target region is adjacent to the target region
  • the ratio of the average depth of field value and the ratio of the white balance/exposure average are both greater than the corresponding threshold, the target area and its adjacent target area are respectively determined as the indoor area and the outdoor area.
  • the optimizing module optimizes the image to be optimized according to the depth of field information of the screen, including:
  • the optimization module calculates, according to the picture depth information, a denoising matrix and a sharpening matrix of each pixel in the image to be optimized, including:
  • the noise matrix and the sharpening matrix calculate a denoising matrix and a sharpening matrix of each pixel.
  • the optimization module normalizes the depth of field value of each pixel point, and obtains a matrix weighting coefficient of each pixel point, including:
  • the pixel points intersecting the edge of the image, Da, Df, Dn are the depth of field values corresponding to the pixel points a, f, and n, respectively, and ⁇ a is the matrix weighting coefficient of the pixel point a.
  • Also disclosed herein is a terminal comprising an image optimization device as described above.
  • This paper provides an image optimization scheme, which acquires the image depth information of the image, and optimizes the image based on the depth information of the image. Since the depth information of the image is proportional to the noise and sharpness of the scene corresponding to each pixel in the image, such an image After the optimization process, the scene close to the human eye has the sharpest sharpness and the smallest noise, which is in line with the human eye observation experience.
  • FIG. 1 is a schematic structural diagram of an image optimization apparatus according to Embodiment 1 of the present invention.
  • FIG. 2 is a flowchart of an image optimization method according to Embodiment 2 of the present invention.
  • Embodiment 3 is a flowchart of an image optimization method according to Embodiment 3 of the present invention.
  • Embodiment 4 is a schematic diagram of an image to be optimized in Embodiment 3 of the present invention.
  • FIG. 5 is a flowchart of an image optimization method according to Embodiment 4 of the present invention.
  • FIG. 6 is a schematic diagram of an image to be optimized in Embodiment 4 of the present invention.
  • Embodiment 1 is a diagrammatic representation of Embodiment 1:
  • FIG. 1 is a schematic structural diagram of an image optimization apparatus according to Embodiment 1 of the present invention. As shown in FIG. 1, in the embodiment, the image optimization apparatus includes the following modules:
  • the obtaining module 11 is configured to obtain the depth of field information of the image to be optimized;
  • the depth of field information refers to the depth of field value of all the pixels in the image, and the depth of field of the pixel is proportional to the distance of the pixel from the camera;
  • the optimization module 12 is configured to optimize the image to be optimized according to the depth of field information.
  • the obtaining module 11 in the above embodiment is configured to measure the image to be optimized by a double camera algorithm, a laser focusing method or a software algorithm, and obtain the depth information of the screen.
  • the binocular algorithm can be used to calculate the depth of field information.
  • the optimization module 12 in the above embodiment is configured to divide the image to be optimized into an indoor area and an outdoor area according to the picture depth information; and use different automatic white balances for the indoor area and the outdoor area respectively. Automatic exposure control.
  • the optimization module 12 in the foregoing embodiment is configured to determine a depth of field value corresponding to each pixel point according to the depth information of the screen, and divide the image to be optimized into at least one depth of field according to the depth of field value corresponding to each pixel point.
  • Target area the difference between the depth values of each pixel in the target area is less than the threshold; calculating the depth-of-field ratio and white balance/exposure ratio of each target area and its adjacent area, when the average depth of field of a target area and adjacent areas
  • the region and the adjacent region are divided into an indoor region and an outdoor region.
  • the optimization module 12 in the foregoing embodiment is configured to calculate a denoising matrix and a sharpening matrix of each pixel in the image to be optimized according to the picture depth information, according to the denoising matrix and the sharpness of each pixel.
  • the matrix is to denoise and sharpen the image of each pixel in the optimized image.
  • the optimization module 12 in the foregoing embodiment is configured to determine a depth value of each pixel according to the depth information of the screen, and normalize the depth value of each pixel to obtain each pixel.
  • the matrix weighting coefficient calculates the denoising matrix and the sharpening matrix of each pixel according to the matrix weighting coefficient and the standard denoising matrix and the sharpening matrix.
  • a terminal is also provided herein, which includes the image optimization device provided herein.
  • the terminal involved in the embodiment of the present invention may be a computer, a mobile computer, a mobile phone, a tablet, or the like.
  • Embodiment 2 is a diagrammatic representation of Embodiment 1:
  • FIG. 2 is a flowchart of an image optimization method according to Embodiment 2 of the present invention.
  • the image optimization method provided by the present invention includes the following steps S201 and S202:
  • the foregoing step S201 includes: measuring a to-be-optimized image by a dual-camera algorithm, a laser focusing method, or a software algorithm to obtain picture depth information.
  • the foregoing step S201 includes: calculating the depth of field information according to the multi-camera.
  • the foregoing step S202 includes: dividing the image to be optimized into an indoor area and an outdoor area according to the picture depth information; and using different automatic white balance and/or automatic exposure control for the indoor area and the outdoor area, respectively.
  • the image to be optimized is segmented according to the picture depth information in the above embodiment.
  • the indoor area and the outdoor area include: determining a depth value corresponding to each pixel according to the depth information of the screen, and dividing the image to be optimized into at least one target area having different depth of field according to the depth value corresponding to each pixel, each of the target areas The difference between the depth values of the pixels is less than the threshold; the depth-of-field ratio and the white balance/exposure ratio of each target area and its adjacent area are calculated, and the ratio of the average depth value of a target area to the adjacent area and the white balance/exposure value are calculated.
  • the ratio is greater than the corresponding threshold, the target area and the adjacent area are divided into an indoor area and an outdoor area.
  • the step S202 includes: calculating a denoising matrix and a sharpening matrix of each pixel in the image to be optimized according to the depth information of the image, and optimizing the image according to the denoising matrix and the sharpening matrix of each pixel The image of each pixel is denoised and sharpened.
  • calculating the denoising matrix and the sharpening matrix of each pixel in the image to be image according to the picture depth information in the foregoing embodiment includes: determining a depth value of each pixel according to the depth information of the picture, for each The depth of field values of the pixels are normalized, the matrix weighting coefficients of each pixel are obtained, and the denoising matrix and the sharpening matrix of each pixel are calculated according to the matrix weighting coefficient and the standard denoising matrix and the sharpening matrix.
  • the terminal is a dual camera phone as an example.
  • Embodiment 3 is a diagrammatic representation of Embodiment 3
  • the present embodiment is directed to a scene in which indoor and outdoor, indoor and outdoor light sources and brightness have large differences in the screen, and quickly locate indoor and outdoor scenes by using depth information and screen statistical information, and perform different automatic white balances for indoor and outdoor scenes (awb, Auto white balance) and automatic exposure (aec, Auto exposure control) processing.
  • the image optimization method provided in this embodiment includes the following steps S301 to S303:
  • S301 sampling double-shooting technology, calculating the depth of field information of the screen by using the left and right camera angle difference;
  • the depth of field information involved in the embodiment refers to a depth of field value of all pixels in the image, and the depth of field value of the pixel is proportional to the distance of the scene of the pixel from the camera.
  • S302 Determine the indoor and outdoor scene boundary by using the depth information output by S101 and the awb/aec statistical information of the screen itself.
  • the step includes: determining a depth of field value corresponding to each pixel point according to the depth information of the screen, and dividing the image to be optimized into at least two or more target areas having different depth of fields according to the depth of field value corresponding to each pixel point.
  • the difference between the depth values of adjacent pixels in the target area is less than a threshold; calculating the ratio of the average depth value of each target area to the adjacent target area and the ratio of the white balance/exposure average, when a target area and phase
  • the ratio of the average depth of field value of the adjacent target area and the ratio of the white balance/exposure average are both greater than the corresponding threshold, the target area and the adjacent target area are respectively determined as the indoor area and the outdoor area.
  • the area 2 is a window
  • the area 1 is an indoor close-up
  • the area 3 is an indoor perspective.
  • the depth of field of the pixel in the area 2 represents the distance from the corresponding scene to the camera. It is much larger than the depth of field of the area 1 and the 3 parts of the area. If you push back, the indoor scene and the outdoor scene can be distinguished according to the average depth of field in each area of the preview screen, because the indoor object has a small depth of field contrast. The contrast between the outdoor scene and the indoor scene is large.
  • the indoor and outdoor scenes are divided.
  • the area 3 generally corresponds to the wall, and the depth value of each pixel is basically the same, so that the area can be used as a target area. .
  • the segmentation needs to meet the following conditions:
  • the average depth of field ratio of different depth of field regions is greater than a threshold T1, such as (average depth of field of region 2 / average depth of field of region 3) > threshold T1;
  • the awb/aec statistical information mean ratio in different depth of field areas is greater than the threshold T2, such as (the awb mean of the area 2 / the awb mean of the area 3) > the threshold T2;
  • T1 and T2 may be set according to the empirical value.
  • the two conditions must be satisfied at the same time. For example, if the depth of field 1 region satisfies the condition 1) the condition 2 is not satisfied, then the depth of field 1 region outdoor region is determined.
  • the gray world and white world algorithm can be sampled simultaneously, and the outdoor white point does not participate in the indoor white balance calculation.
  • Embodiment 4 is a diagrammatic representation of Embodiment 4:
  • the present embodiment is directed to a scene in which the depth of field in the picture extends linearly from near to far.
  • the linear depth of field value as a linear denoising weight value
  • the denoising intensity is adjusted, and the denoising intensity is gradually strengthened from near to far.
  • the sharpening parameters are gradually weakened, which is in line with the human eye observation experience.
  • the image optimization method provided in this embodiment includes the following steps S501 to 503:
  • S501 sampling double-shooting technology, calculating the depth of field information of the image by using the left and right camera angle difference, and normalizing the depth value of each pixel of the image to obtain the matrix coefficient of each pixel;
  • the double-shooting technology calculates the depth of field information of the picture by using the left and right camera angle difference, as shown in FIG. 6, normalizing the depth value of each pixel of the image to obtain the matrix coefficient ⁇ :
  • S502 Calculate a denoising matrix and a sharpening matrix of each pixel point
  • the depth information calculated in step S501 is used as the weight ⁇ , and the standard denoising matrix A and the sharpening matrix B are respectively multiplied:
  • step S502 performing denoising and sharpening on the picture P by using the denoising and sharpening matrix obtained in step S502, where
  • the same image may include an indoor/outdoor window, and an extended scene such as a road, that is, an application scenario of the third embodiment and the fourth embodiment.
  • the image optimization methods provided in the third embodiment and the fourth embodiment are sequentially executed.
  • An embodiment of the present invention provides an image optimization scheme, which acquires image depth information of an image, and optimizes the image based on the depth information of the image. Since the depth information of the image is proportional to the brightness of the scene corresponding to each pixel of the image, the image is optimized. After treatment, it is in line with human eye observation experience.

Abstract

一种图像优化方法、装置及终端,该方法包括:获取待优化图像的画面景深信息;根据画面景深信息对待优化图像进行优化。

Description

一种图像优化方法、装置及终端 技术领域
本文涉及但不限于图像处理领域,尤其涉及一种图像优化方法、装置及终端。
背景技术
在拍照、或者后期处理图片时,当画面出现室内外光源不同强度不同的场景,相关技术的处理方法以画面主体景物为主,比如主体为室内场景,则牺牲窗外景物的优化效果,室外场景会出现过曝和偏色。在相关技术中,技术人员提出可以通过将室外景物和室内景物权衡调试以减轻室外场景的过曝和偏色程度,但是调节室内室外场景的权重必然会导致牺牲主体场景的部分优化效果,室内外优化效果很难做到比较好的平衡。
发明内容
以下是对本文详细描述的主题的概述。本概述并非是为了限制权利要求的保护范围。
本文提供一种图像优化方法、装置及终端,可以解决相关技术中图像优化方法以主体为准进行全图像优化存在的牺牲背景优化效果的问题。
本文提供了一种图像优化方法,其包括:
获取待优化图像的画面景深信息;
根据所述画面景深信息对所述待优化图像进行优化。
可选地,上述图像优化方法中,所述获取待优化图像的画面景深信息包括:
通过双摄算法、激光对焦方法或软件算法测量所述待优化图像,获取所述画面景深信息。
可选地,上述图像优化方法中,所述根据所述画面景深信息对所述待优化图像进行优化包括:
根据所述画面景深信息将所述待优化图像分割为室内区域和室外区域;
针对室内区域和室外区域,分别使用不同的自动白平衡和/或自动曝光控制。
可选地,上述图像优化方法中,所述根据所述画面景深信息将所述待优化图像分割为室内区域和室外区域包括:
根据所述画面景深信息确定每个像素点对应的景深值,根据每个像素点对应的景深值将所述待优化图像分割为两个或两个以上的目标区域,同一个目标区域内相邻像素点的景深值的差值小于阈值;计算每个目标区域与其相邻的目标区域的平均景深值的比值及白平衡/曝光平均值的比值,当一目标区域与其相邻的目标区域的平均景深值的比值及白平衡/曝光平均值的比值均大于对应阈值时,将该目标区域及其相邻的目标区域分别确定为室内区域和室外区域。
可选地,上述图像优化方法中,所述根据所述画面景深信息对所述待优化图像进行优化包括:
根据所述画面景深信息计算所述待优化图像内每个像素点的去噪矩阵及锐化矩阵,根据所述每个像素点的去噪矩阵及锐化矩阵对所述待优化图像内每个像素点的图像进行去噪及锐化处理。
可选地,上述图像优化方法中,所述根据所述画面景深信息计算所述待优化图像内每个像素点的去噪矩阵及锐化矩阵包括:
根据所述画面景深信息确定每个像素点的景深值,对所述每个像素点的景深值进行归一化处理,获取每个像素点的矩阵加权系数,根据所述矩阵加权系数及标准去噪矩阵及锐化矩阵,计算所述每个像素点的去噪矩阵及锐化矩阵。
可选地,上述图像优化方法中,所述对所述每个像素点的景深值进行归一化处理,获取每个像素点的矩阵加权系数包括:
使用γa=Da/(Df-Dn)进行归一化处理获取每个像素点的矩阵加权系 数,其中,a为图像中任一像素点,n和f为穿过a、且垂直图像边缘的直线与图像边缘相交的像素点,Da、Df、Dn分别为像素点a、f、n对应的景深值,γa为像素点a的矩阵加权系数。
本文还公开了一种图像优化装置,包括:
获取模块,设置为获取待优化图像的画面景深信息;
优化模块,设置为根据所述画面景深信息对所述待优化图像进行优化。
可选地,上述图像优化装置中,所述获取模块获取待优化图像的画面景深信息包括:
通过双摄算法、激光对焦方法或软件算法测量所述待优化图像,获取所述画面景深信息。
可选地,上述图像优化装置中,所述优化模块根据所述画面景深信息对所述待优化图像进行优化包括:
根据所述画面景深信息将所述待优化图像分割为室内区域和室外区域;
针对室内区域和室外区域,分别使用不同的自动白平衡和/或自动曝光控制。
可选地,上述图像优化装置中,所述优化模块根据所述画面景深信息将所述待优化图像分割为室内区域和室外区域包括:
根据所述画面景深信息确定每个像素点对应的景深值,根据每个像素点对应的景深值将所述待优化图像分割为至少两个或两个以上的目标区域,每个目标区域内相邻像素点的景深值的差值小于阈值;计算每个目标区域与其相邻的目标区域的平均景深值的比值及白平衡/曝光平均值的比值,当一目标区域与其相邻的目标区域的平均景深值的比值及白平衡/曝光平均值的比值均大于对应阈值时,将所述目标区域及其相邻的目标区域分别确定为室内区域和室外区域。
可选地,上述图像优化装置中,所述优化模块根据所述画面景深信息对所述待优化图像进行优化包括:
根据所述画面景深信息计算所述待优化图像内每个像素点的去噪矩阵及 锐化矩阵,根据所述每个像素点的去噪矩阵及锐化矩阵对所述待优化图像内每个像素点的图像进行去噪及锐化处理。
可选地,上述图像优化装置中,所述优化模块根据所述画面景深信息计算所述待优化图像内每个像素点的去噪矩阵及锐化矩阵包括:
根据所述画面景深信息确定每个像素点的景深值,对所述每个像素点的景深值进行归一化处理,获取每个像素点的矩阵加权系数,根据所述矩阵加权系数及标准去噪矩阵及锐化矩阵,计算所述每个像素点的去噪矩阵及锐化矩阵。
可选地,上述图像优化装置中,所述优化模块对所述每个像素点的景深值进行归一化处理,获取每个像素点的矩阵加权系数包括:
使用γa=Da/(Df-Dn)进行归一化处理获取每个像素点的矩阵加权系数,其中,a为图像中任一像素点,n和f为穿过a、且垂直图像边缘的直线与图像边缘相交的像素点,Da、Df、Dn分别为像素点a、f、n对应的景深值,γa为像素点a的矩阵加权系数。
本文还公开了一种终端,包括如上所述的图像优化装置。
本文提供了一种图像优化方案,获取图像的画面景深信息,基于画面景深信息对图像进行优化处理,由于画面景深信息是与图像中每个像素点对应景物的噪声及锐度成正比,这样图像优化处理后,靠近人眼的景物锐度最大、噪声最小,符合人眼观察经验。
在阅读并理解了附图和详细描述后,可以明白其他方面。
附图概述
图1为本发明实施例一提供的图像优化装置的结构示意图;
图2为本发明实施例二提供的图像优化方法的流程图;
图3是本发明实施例三提供的图像优化方法的流程图;
图4是本发明实施例三中的待优化图像的示意图;
图5是本发明实施例四提供的图像优化方法的流程图;
图6是本发明实施例四中的待优化图像的示意图。
本发明的实施方式
下文中将结合附图对本文的实施例进行详细说明。需要说明的是,在不冲突的情况下,本申请中的实施例及实施例中的特征可以相互任意组合。
实施例一:
图1为本发明实施例一提供的图像优化装置的结构示意图,由图1可知,在本实施例中,图像优化装置包括如下模块:
获取模块11,设置为获取待优化图像的画面景深信息;画面景深信息是指图像内所有像素点的景深值,像素点的景深值与像素点的景物距离摄像头的距离的成正比;
优化模块12,设置为根据画面景深信息对待优化图像进行优化。
在可选实施例中,上述实施例中的获取模块11设置为通过双摄算法、激光对焦方法或软件算法测量待优化图像,获取画面景深信息。在实际应用中,若终端具备多个摄像头,在拍照时,可以使用双摄算法计算画面景深信息。
在可选实施例中,上述实施例中的优化模块12设置为根据画面景深信息将待优化图像分割为室内区域和室外区域;针对室内区域和室外区域,分别使用不同的自动白平衡和/或自动曝光控制。
在可选实施例中,上述实施例中的优化模块12设置为根据画面景深信息确定每个像素点对应的景深值,根据每个像素点对应的景深值将待优化图像分割为至少一个景深不同的目标区域,目标区域内每个像素点的景深值的差值小于阈值;计算每个目标区域与其相邻区域的景深比值及白平衡/曝光比值,当一目标区域与相邻区域的平均景深值的比值及白平衡/曝光值的比值均大于对应阈值时,将区域及相邻区域分割为室内区域和室外区域。
在可选实施例中,上述实施例中的优化模块12设置为根据画面景深信息计算待优化图像内每个像素点的去噪矩阵及锐化矩阵,根据每个像素点的去噪矩阵及锐化矩阵对待优化图像内每个像素点的图像进行去噪及锐化处理。
在可选实施例中,上述实施例中的优化模块12设置为根据画面景深信息确定每个像素点的景深值,对每个像素点的景深值进行归一化处理,获取每个像素点的矩阵加权系数,根据矩阵加权系数及标准去噪矩阵及锐化矩阵,计算每个像素点的去噪矩阵及锐化矩阵。
在可选实施例中,上述实施例中的优化模块12设置为使用公式γa=Da/(Df-Dn)进行归一化处理获取每个像素点的矩阵加权系数,其中,a为图像中任一像素点,n和f为穿过a、且垂直图像边缘的直线与图像边缘相交的像素点,Da、Df、Dn分别为像素点a、f、n对应的景深值,γa为像素点a的矩阵加权系数。
对应的,本文还提供了一种终端,其包括本文提供的图像优化装置。在实际应用中,本发明实施例所涉及的终端可以是电脑、移动电脑、手机、平板等。
实施例二:
图2为本发明实施例二提供的图像优化方法的流程图,由图2可知,在本实施例中,本发明提供的图像优化方法包括如下步骤S201和S202:
S201:获取待优化图像的画面景深信息;
S202:根据画面景深信息对待优化图像进行优化。
在可选实施例中,上述步骤S201包括:通过双摄算法、激光对焦方法或软件算法测量待优化图像,获取画面景深信息。
在可选实施例中,当设备包括至少两个摄像头时,上述步骤S201包括:根据多摄像头计算画面景深信息。
在可选实施例中,上述步骤S202包括:根据画面景深信息将待优化图像分割为室内区域和室外区域;针对室内区域和室外区域,分别使用不同的自动白平衡和/或自动曝光控制。
在可选实施例中,上述实施例中的根据画面景深信息将待优化图像分割 为室内区域和室外区域包括:根据画面景深信息确定每个像素点对应的景深值,根据每个像素点对应的景深值将待优化图像分割为至少一个景深不同的目标区域,目标区域内每个像素点的景深值的差值小于阈值;计算每个目标区域与其相邻区域的景深比值及白平衡/曝光比值,当一目标区域与相邻区域的平均景深值的比值及白平衡/曝光值的比值均大于对应阈值时,将目标区域及相邻区域分割为室内区域和室外区域。
在可选实施例中,上述步骤S202包括:根据画面景深信息计算待优化图像内每个像素点的去噪矩阵及锐化矩阵,根据每个像素点的去噪矩阵及锐化矩阵对待优化图像内每个像素点的图像进行去噪及锐化处理。
在可选实施例中,上述实施例中的根据画面景深信息计算待化图像内每个像素点的去噪矩阵及锐化矩阵包括:根据画面景深信息确定每个像素点的景深值,对每个像素点的景深值进行归一化处理,获取每个像素点的矩阵加权系数,根据矩阵加权系数及标准去噪矩阵及锐化矩阵,计算每个像素点的去噪矩阵及锐化矩阵。
在可选实施例中,上述实施例中的对每个像素点的景深值进行归一化处理,获取每个像素点的矩阵加权系数包括:使用公式γa=Da/(Df-Dn)进行归一化处理获取每个像素点的矩阵加权系数,其中,a为图像中任一像素点,n和f为穿过a、且垂直图像边缘的直线与图像边缘相交的像素点,Da、Df、Dn分别为像素点a、f、n对应的景深值,γa为像素点a的矩阵加权系数。
现结合实际应用场景对本发明实施例做诠释说明。
在以下实施例中,以终端为双摄像头手机为例进行说明。
实施例三:
本实施例针对画面中同时出现室内室外,室内外的光源、亮度差别较大的场景,通过使用景深信息与画面统计信息迅速定位室内外场景,针对室内外场景做不同的自动白平衡(awb,Auto white balance)和自动曝光(aec,Auto exposure control)处理。
如图3所示,本实施例提供的图像优化方法包括以下步骤S301到S303:
S301:采样双摄技术,利用左右摄像头视角差计算画面景深信息;
可选地,本实施例中涉及的画面景深信息是指图像内所有像素点的景深值,像素点的景深值与像素点的景物距离摄像头的距离的成正比。
S302:利用S101输出的景深信息与画面本身的awb/aec统计信息,确定室内外场景边界。
可选地,本步骤包括:根据画面景深信息确定每个像素点对应的景深值,根据每个像素点对应的景深值将待优化图像分割为至少两个或两个以上的景深不同的目标区域,目标区域内相邻像素点的景深值的差值小于阈值;计算每个目标区域与其相邻的目标区域的平均景深值的比值及白平衡/曝光平均值的比值,当一目标区域与相邻的目标区域的平均景深值的比值及白平衡/曝光平均值的比值均大于对应阈值时,将该目标区域及相邻的目标区域分别确定为室内区域和室外区域。
如图4所示,区域2部分为窗户,区域1部分为室内近景,区域3部分为室内远景,在实际应用中,区域2部分内的像素点的景深值代表着其对应景物到摄像头的距离远大于区域1部分与区域3部分的像素点的景深值,那么反推的,根据预览画面内每个区域的平均景深值也可以将室内景物及室外景物分辨开,因为室内物体景深反差较小,室外景物与室内景物景深反差较大,边界阈值处理后分割室内外场景,例如区域3部分一般对应着墙壁,其每个像素点的景深值基本一样,就可以将这样的区域作为一个目标区域。
可选地,在实际应用中,分割需要满足如下条件:
条件1)不同景深区域(即所述目标区域和相邻区域)的平均景深比值大于阈值T1,如(区域2的平均景深/区域3的平均景深)>阈值T1;
条件2)不同景深区域awb/aec统计信息均值比值大于阈值T2,如(区域2的awb均值/区域3的awb均值)>阈值T2;
可选地,可以根据经验值,设置T1和T2,在本实施例中,两个条件必须同时满足,如景深1区域满足条件1)不满足条件2)则判断景深1区域室外区域。
S303:对步骤S102判定的室内外区域分别采样不同的awb与aec算法处理。
可选地,针对awb,可以同时采样灰世界与白世界算法处理,室外的白点不参与室内的白平衡计算。
实施例四:
本实施例针对画面中有近处到远处景深呈线性延伸的场景,通过将画面线性的景深值作为一个线性的去噪权重值,调节去噪强度,从近到远,去噪强度逐渐加强,锐化参数逐渐减弱,符合人眼观察经验。
如图5所示,本实施例提供的图像优化方法包括以下步骤S501至503:
S501:采样双摄技术,利用左右摄像头视角差计算画面景深信息,将图像每个像素点景深值做归一化得到每个像素点的矩阵系数;
可选地,采样相关技术中双摄技术,利用左右摄像头视角差计算画面景深信息,如图6所示,将图像每个像素点景深值做归一化得到矩阵系数γ:
可选地,采用公式γa=Da/(Df-Dn)对图像每个像素点景深值做归一化得到矩阵系数γ,其中,a为图像中任一点,n和f为穿过a、且垂直图像边缘的直线与图像边缘的交点,Da、Df、Dn分别为点a、f、n对应的景深值。
S502:计算每个像素点的去噪矩阵及锐化矩阵;
可选地,利用步骤S501计算的深度信息作为权重γ,分别乘以标准的去噪矩阵A与锐化矩阵B:
其中,a点的去噪矩阵A’=γa*A;a点的锐化矩阵B’=(1-γa)*B;
S503:进行优化处理;
可选地,利用步骤S502得出的去噪和锐化矩阵对画面P进行去噪和锐化,其中,
Figure PCTCN2016088609-appb-000001
可选地,在实际应用中,同一图像中,可能在包括室内室外窗口的同时,还包括马路等延伸场景,即同时包括实施例三及实施例四的运用场景,此时,可以分别的、依次执行实施例三及实施例四提供的图像优化方法。
本领域普通技术人员可以理解上述方法中的全部或部分步骤可通过程序来指令相关硬件(例如处理器)完成,所述程序可以存储于计算机可读存储 介质中,如只读存储器、磁盘或光盘等。可选地,上述实施例的全部或部分步骤也可以使用一个或多个集成电路来实现。相应地,上述实施例中的模块/单元可以采用硬件的形式实现,例如通过集成电路来实现其相应功能,也可以采用软件功能模块的形式实现,例如通过处理器执行存储于存储器中的程序指令来实现其相应功能。本申请不限制于任何特定形式的硬件和软件的结合。
工业实用性
本发明实施例提供了一种图像优化方案,获取图像的画面景深信息,基于画面景深信息对图像进行优化处理,由于画面景深信息是与图像每个像素点对应景物的亮度成正比,这样图像优化处理后,符合人眼观察经验。

Claims (15)

  1. 一种图像优化方法,其特征在于,包括:
    获取待优化图像的画面景深信息;
    根据所述画面景深信息对所述待优化图像进行优化。
  2. 如权利要求1所述的图像优化方法,其中,所述获取待优化图像的画面景深信息包括:
    通过双摄算法、激光对焦方法或软件算法测量所述待优化图像,获取所述画面景深信息。
  3. 如权利要求1或2所述的图像优化方法,其中,所述根据所述画面景深信息对所述待优化图像进行优化包括:
    根据所述画面景深信息将所述待优化图像分割为室内区域和室外区域;
    针对室内区域和室外区域,分别使用不同的自动白平衡和/或自动曝光控制。
  4. 如权利要求3所述的图像优化方法,其特征在于,所述根据所述画面景深信息将所述待优化图像分割为室内区域和室外区域包括:
    根据所述画面景深信息确定每个像素点对应的景深值,根据每个像素点对应的景深值将所述待优化图像分割为两个或两个以上的目标区域,同一个目标区域内相邻像素点的景深值的差值小于阈值;计算每个目标区域与其相邻的目标区域的平均景深值的比值及白平衡/曝光平均值的比值,当一目标区域与其相邻的目标区域的平均景深值的比值及白平衡/曝光平均值的比值均大于对应阈值时,将该目标区域及其相邻的目标区域分别确定为室内区域和室外区域。
  5. 如权利要求4所述的图像优化方法,其中,所述根据所述画面景深信息对所述待优化图像进行优化包括:
    根据所述画面景深信息计算所述待优化图像内每个像素点的去噪矩阵及锐化矩阵,根据所述每个像素点的去噪矩阵及锐化矩阵对所述待优化图像内 每个像素点的图像进行去噪及锐化处理。
  6. 如权利要求5所述的图像优化方法,其中,所述根据所述画面景深信息计算所述待优化图像内每个像素点的去噪矩阵及锐化矩阵包括:
    根据所述画面景深信息确定每个像素点的景深值,对所述每个像素点的景深值进行归一化处理,获取每个像素点的矩阵加权系数,根据所述矩阵加权系数及标准去噪矩阵及锐化矩阵,计算所述每个像素点的去噪矩阵及锐化矩阵。
  7. 如权利要求6所述的图像优化方法,其特征在于,所述对所述每个像素点的景深值进行归一化处理,获取每个像素点的矩阵加权系数包括:
    使用γa=Da/(Df-Dn)进行归一化处理获取每个像素点的矩阵加权系数,其中,a为图像中任一像素点,n和f为穿过a、且垂直图像边缘的直线与图像边缘相交的像素点,Da、Df、Dn分别为像素点a、f、n对应的景深值,γa为像素点a的矩阵加权系数。
  8. 一种图像优化装置,包括:
    获取模块,设置为获取待优化图像的画面景深信息;
    优化模块,设置为根据所述画面景深信息对所述待优化图像进行优化。
  9. 如权利要求8所述的图像优化装置,其中,所述获取模块获取待优化图像的画面景深信息包括:
    通过双摄算法、激光对焦方法或软件算法测量所述待优化图像,获取所述画面景深信息。
  10. 如权利要求8所述的图像优化装置,其中,所述优化模块根据所述画面景深信息对所述待优化图像进行优化包括:
    根据所述画面景深信息将所述待优化图像分割为室内区域和室外区域;
    针对室内区域和室外区域,分别使用不同的自动白平衡和/或自动曝光控制。
  11. 如权利要求10所述的图像优化装置,其中,所述优化模块根据所述画面景深信息将所述待优化图像分割为室内区域和室外区域包括:
    根据所述画面景深信息确定每个像素点对应的景深值,根据每个像素点对应的景深值将所述待优化图像分割为至少两个或两个以上的目标区域,每个目标区域内相邻像素点的景深值的差值小于阈值;计算每个目标区域与其相邻的目标区域的平均景深值的比值及白平衡/曝光平均值的比值,当一目标区域与其相邻的目标区域的平均景深值的比值及白平衡/曝光平均值的比值均大于对应阈值时,将所述目标区域及其相邻的目标区域分别确定为室内区域和室外区域。
  12. 如权利要求8至11任一项所述的图像优化装置,其中,所述优化模块根据所述画面景深信息对所述待优化图像进行优化包括:
    根据所述画面景深信息计算所述待优化图像内每个像素点的去噪矩阵及锐化矩阵,根据所述每个像素点的去噪矩阵及锐化矩阵对所述待优化图像内每个像素点的图像进行去噪及锐化处理。
  13. 如权利要求12所述的图像优化装置,其中,所述优化模块根据所述画面景深信息计算所述待优化图像内每个像素点的去噪矩阵及锐化矩阵包括:
    根据所述画面景深信息确定每个像素点的景深值,对所述每个像素点的景深值进行归一化处理,获取每个像素点的矩阵加权系数,根据所述矩阵加权系数及标准去噪矩阵及锐化矩阵,计算所述每个像素点的去噪矩阵及锐化矩阵。
  14. 如权利要求13所述的图像优化装置,其中,所述优化模块对所述每个像素点的景深值进行归一化处理,获取每个像素点的矩阵加权系数包括:
    使用γa=Da/(Df-Dn)进行归一化处理获取每个像素点的矩阵加权系数,其中,a为图像中任一像素点,n和f为穿过a、且垂直图像边缘的直线与图像边缘相交的像素点,Da、Df、Dn分别为像素点a、f、n对应的景深值,γa为像素点a的矩阵加权系数。
  15. 一种终端,包括如权利要求8至14任一项所述的图像优化装置。
PCT/CN2016/088609 2016-05-05 2016-07-05 一种图像优化方法、装置及终端 WO2017190415A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US16/097,282 US20190139198A1 (en) 2016-05-05 2016-07-05 Image Optimization Method and Device, and Terminal
EP16900937.0A EP3438921A4 (en) 2016-05-05 2016-07-05 IMAGE OPTIMIZATION METHOD AND DEVICE, AND TERMINAL THEREOF

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610293530.9 2016-05-05
CN201610293530.9A CN107346531A (zh) 2016-05-05 2016-05-05 一种图像优化方法、装置及终端

Publications (1)

Publication Number Publication Date
WO2017190415A1 true WO2017190415A1 (zh) 2017-11-09

Family

ID=60202546

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/088609 WO2017190415A1 (zh) 2016-05-05 2016-07-05 一种图像优化方法、装置及终端

Country Status (4)

Country Link
US (1) US20190139198A1 (zh)
EP (1) EP3438921A4 (zh)
CN (1) CN107346531A (zh)
WO (1) WO2017190415A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107220953A (zh) * 2017-06-16 2017-09-29 广东欧珀移动通信有限公司 图像处理方法、装置及终端
CN108182667B (zh) * 2017-12-29 2020-07-17 珠海大横琴科技发展有限公司 一种图像优化方法、终端及计算机可读存储介质
CN109605429B (zh) * 2018-12-06 2020-10-09 蒋玉素 可清洗型弯头式去胡刀
CN113570650B (zh) * 2020-04-28 2024-02-02 合肥美亚光电技术股份有限公司 景深的判断方法、装置、电子设备和存储介质
CN113965663A (zh) * 2020-07-21 2022-01-21 深圳Tcl新技术有限公司 一种图像画质优化方法、智能终端及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101267505A (zh) * 2008-04-25 2008-09-17 北京中星微电子有限公司 一种曝光时间调整方法、装置及一种摄像头
CN102999901A (zh) * 2012-10-17 2013-03-27 中国科学院计算技术研究所 基于深度传感器的在线视频分割后的处理方法及系统
US20150170389A1 (en) * 2013-12-13 2015-06-18 Konica Minolta Laboratory U.S.A., Inc. Automatic selection of optimum algorithms for high dynamic range image processing based on scene classification
CN104794688A (zh) * 2015-03-12 2015-07-22 北京航空航天大学 基于深度信息分离天空区域的单幅图像去雾方法及装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7425984B2 (en) * 2003-04-04 2008-09-16 Stmicroelectronics, Inc. Compound camera and methods for implementing auto-focus, depth-of-field and high-resolution functions
US20070093993A1 (en) * 2005-10-20 2007-04-26 Stork David G End-to-end design of electro-optic imaging systems using backwards ray tracing from the detector to the source

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101267505A (zh) * 2008-04-25 2008-09-17 北京中星微电子有限公司 一种曝光时间调整方法、装置及一种摄像头
CN102999901A (zh) * 2012-10-17 2013-03-27 中国科学院计算技术研究所 基于深度传感器的在线视频分割后的处理方法及系统
US20150170389A1 (en) * 2013-12-13 2015-06-18 Konica Minolta Laboratory U.S.A., Inc. Automatic selection of optimum algorithms for high dynamic range image processing based on scene classification
CN104794688A (zh) * 2015-03-12 2015-07-22 北京航空航天大学 基于深度信息分离天空区域的单幅图像去雾方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3438921A4 *

Also Published As

Publication number Publication date
CN107346531A (zh) 2017-11-14
EP3438921A1 (en) 2019-02-06
US20190139198A1 (en) 2019-05-09
EP3438921A4 (en) 2019-05-08

Similar Documents

Publication Publication Date Title
US10997696B2 (en) Image processing method, apparatus and device
WO2019105262A1 (zh) 背景虚化处理方法、装置及设备
CN111028189B (zh) 图像处理方法、装置、存储介质及电子设备
JP6027159B2 (ja) 画像ぼかし方法および機器、および電子装置
WO2017190415A1 (zh) 一种图像优化方法、装置及终端
WO2018228467A1 (zh) 图像曝光方法、装置、摄像设备及存储介质
US9992478B2 (en) Image processing apparatus, image pickup apparatus, image processing method, and non-transitory computer-readable storage medium for synthesizing images
WO2019105261A1 (zh) 背景虚化处理方法、装置及设备
WO2019105254A1 (zh) 背景虚化处理方法、装置及设备
KR102049080B1 (ko) 영상 처리 장치 및 방법
US11282176B2 (en) Image refocusing
US10235745B2 (en) Image processing method, computer storage medium, apparatus and terminal
US11871123B2 (en) High dynamic range image synthesis method and electronic device
JP6624785B2 (ja) 画像処理方法、画像処理装置、撮像装置、プログラム、および、記憶媒体
US20240022702A1 (en) Foldable electronic device for multi-view image capture
US20160275657A1 (en) Imaging apparatus, image processing apparatus and method of processing image
WO2021145913A1 (en) Estimating depth based on iris size
CN113965664B (zh) 一种图像虚化方法、存储介质以及终端设备
JP2019075716A (ja) 画像処理装置、画像処理方法、及びプログラム
JP2018181070A (ja) 画像処理装置および画像処理方法
JP6949494B2 (ja) 画像処理装置および画像処理方法、撮像装置、プログラム
JP2017182668A (ja) データ処理装置、撮像装置、及びデータ処理方法
WO2021168749A1 (en) Dynamic adjustment of a region of interest for image capture
JP7297566B2 (ja) 画像処理装置、撮像装置、画像処理方法およびプログラム
JP2017011366A (ja) 画像処理装置および画像処理方法ならびに撮像装置

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2016900937

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2016900937

Country of ref document: EP

Effective date: 20181029

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16900937

Country of ref document: EP

Kind code of ref document: A1