WO2020259603A1 - 图像处理装置及图像处理方法 - Google Patents

图像处理装置及图像处理方法 Download PDF

Info

Publication number
WO2020259603A1
WO2020259603A1 PCT/CN2020/098169 CN2020098169W WO2020259603A1 WO 2020259603 A1 WO2020259603 A1 WO 2020259603A1 CN 2020098169 W CN2020098169 W CN 2020098169W WO 2020259603 A1 WO2020259603 A1 WO 2020259603A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
image
image processing
pixel
target object
Prior art date
Application number
PCT/CN2020/098169
Other languages
English (en)
French (fr)
Inventor
徳永将之
Original Assignee
海信视像科技股份有限公司
东芝视频解决方案株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 海信视像科技股份有限公司, 东芝视频解决方案株式会社 filed Critical 海信视像科技股份有限公司
Priority to CN202080002690.8A priority Critical patent/CN112470165B/zh
Publication of WO2020259603A1 publication Critical patent/WO2020259603A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/162—Detection; Localisation; Normalisation using pixel segmentation or colour matching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
    • G06V10/993—Evaluation of the quality of the acquired pattern

Definitions

  • the embodiments of the present application relate to an image processing device and an image processing method.
  • Patent Document 1 Japanese Patent Application Publication No. 2019-40382
  • the purpose of the embodiments of the present application is to provide an image processing device and an image processing method that can perform effective image quality adjustment processing for the target by performing target detection based on machine learning and using color space information.
  • the image processing device includes: a reduction unit that reduces an input image to output a reduced image; an object detection unit that detects a predetermined target object from the reduced image; and an area determination unit based on The detection result of the target detection unit determines the target candidate region including the target object in the input image; the color space determination unit determines the target candidate region based on the information of the color space corresponding to the target object Whether it is a region corresponding to the target object; and an image processing circuit that controls image processing of the input image based on the determination result of the color space determination unit.
  • Fig. 1 is a block diagram showing an image processing device according to an embodiment of the present application
  • FIG. 2 is an explanatory diagram for explaining an example of processing of the target detection unit 4;
  • FIG. 3 is an explanatory diagram for explaining an example of processing of the target detection unit 4;
  • Fig. 4 is a flowchart for explaining the operation of the embodiment.
  • FIG. 1 is a block diagram showing an image processing device according to an embodiment of the present application.
  • a detector is used to determine a target in a moving image, and the color space of the detected target area is determined, thereby accurately controlling the image quality improvement process for the target or its vicinity.
  • the detector Use an inductive model based on machine learning. As a result, the image quality of objects such as human faces in the image can be improved.
  • the image processing device of this embodiment can be used in various devices that perform image processing.
  • the image processing device of this embodiment can be used in television receivers, video recorders, etc., to improve various target image quality in images of broadcast programs, and as a result, it is possible to obtain high-quality moving images on the entire image.
  • the image processing device of this embodiment can also be used for surveillance cameras, vehicle-mounted cameras, etc., to improve the image quality of various objects in the captured moving images, and as a result, the recognition accuracy of objects such as humans can also be improved.
  • the input image is supplied to the reduction circuit 1, the area determination circuit 2, and the image quality improvement processing circuit 3.
  • the input image is a moving image based on a predetermined frame rate, a predetermined resolution, and a predetermined standard. For example, it may be a moving image based on a broadcast signal received by a television receiver or the like, or it may be a moving image obtained from a predetermined camera system.
  • the reduction circuit 1 as a reduction unit performs reduction processing on the input image.
  • the reduction circuit 1 may employ various reduction algorithms such as known bilinear and bicubic methods, and the algorithm is not particularly limited.
  • the reduction circuit 1 obtains a reduced image from the input image. It should be noted that the reduction magnification depends on the input image size and the calculation speed of the target detection unit 4.
  • the reduction circuit 1 sequentially outputs reduced images generated at a predetermined frame rate to the target detection unit 4.
  • the object detection unit 4 uses machine learning technology to perform processing of detecting an object of the detection target (hereinafter referred to as a target object) from the input reduced image. It should be noted that the target object may also be a predetermined target.
  • a predetermined network for constructing an inference model for target object detection is composed of hardware or software.
  • the inference model of the target detection unit 4 a large amount of training data created by attaching information indicating the range of the target object in the reduced image as a label to the reduced image is provided to a predetermined network for learning, thereby obtaining the inference model.
  • This inductive model outputs the information indicating the range of the target object together with the reliability information for the input of the reduced image.
  • DNN Deep Neural Network
  • the target detection unit 4 may also use methods other than deep neural networks, for example, methods such as Haar-Like.
  • 2 and 3 are explanatory diagrams for explaining an example of processing of the target detection unit 4.
  • 2 and 3 show examples of detection processing when the target object is a human face.
  • the reduced image Pin in FIGS. 2 and 3 represents the reduced image input to the target detection unit 4.
  • the reduced image Pin contains images of the persons O1 and O2, and the circle represents the image of the target face.
  • the target detection unit 4 uses generalization processing to set the region DR1 including the face portion of the person O1 and the region DR2 including the face portion of the person O2 as the target object as shown in the reduced image Pout.
  • the detection area is tested.
  • the target detection unit 4 detects a face part, and uses a rectangular area of a predetermined size centered on the coordinates of the center of the detected face part as the detection area.
  • the target detection unit 4 outputs the information about the regions DR1 and DR2 as the detection result of the target object.
  • FIG. 3 shows an example in which the range of the target object is detected in accordance with a small area (hereinafter referred to as a determination small area) divided into a reduced image Pin by a grid.
  • the induction model constituting the target detection unit 4
  • the induction model can be acquired by learning a reduced image to which a label indicating whether it is a target object is added for each determination small area as training data.
  • the target detection unit 4 detects the area DR3 and the area DR4 as the detection area of the target object through the induction process, as shown in the reduced image Pout, wherein the area DR3 includes the 2 detected as the face part of the person O1. A small determination area, and the area DR4 includes four small determination areas detected as a face part of the person O2.
  • the target detection unit 4 outputs information related to the areas DR3 and DR4 as the detection result of the target object.
  • the target detection unit 4 outputs information related to the detection area to the area determination circuit 2.
  • the area determining circuit 2 as the area determining unit converts the detection area detected on the reduced image into an area of a position and size corresponding to the size of the input image in either case in the examples of FIGS. 2 and 3 ( Hereinafter referred to as target induction area).
  • the target detection unit 4 obtains a candidate for a region considered to constitute a target object (hereinafter referred to as a target candidate region) for the input image of the target summary region. For example, the target detection unit 4 determines whether the input image of the target summary area is a pixel in the target candidate area, that is, a candidate for the pixels constituting the target object (hereinafter referred to as target pixel candidate) for each pixel in the target summary area. ).
  • the area determination circuit 2 may also use a score of reliability in determining the detection area as a score for determining whether each pixel of the target summary area is a target pixel candidate (hereinafter referred to as an area score).
  • an area score a score for determining whether each pixel of the target summary area is a target pixel candidate.
  • all pixels in the target summary area corresponding to the area DR1 have the same area score
  • all pixels in the target summary area corresponding to the area DR2 have the same area score.
  • all the pixels in the target summary area have the same area score.
  • the area determination circuit 2 may not only use the reliability score during the determination of the detection area, but also use other information to determine the area score.
  • the area determination circuit 2 may also use pixels whose area scores exceed a predetermined threshold as target pixel candidates.
  • the target pixel candidate is provided to the color space determination unit 5. It should be noted that the target pixel is a pixel for image processing using processing parameters for the target object.
  • the color space determination unit 5 determines the target pixel based on whether the pixel of the target pixel candidate holds information corresponding to the color space of the target object. For example, when the target object is a human face, and when the color information of the pixel of the target pixel candidate indicates the human skin color (face color), it can be determined that the pixel holds information corresponding to the color space of the target object.
  • the color space determination unit 5 may also convert each pixel of the target pixel candidate in the input image into information of a predetermined color space to determine its color. For example, the color space determination unit 5 converts each pixel of the target pixel candidate in the input image into the HSV color space, and determines for each pixel whether the color of the pixel exists in a predetermined range corresponding to the color of the target object in the HSV color space (Hereinafter referred to as the target color range) to determine the target pixel. In addition, it is also possible to determine the target pixel based on whether at least one of hue (H), saturation (S), and brightness (V) in the HSV color space exists in the target color range.
  • H hue
  • S saturation
  • V brightness
  • the color space determination unit 5 can also determine whether the color of the pixel exists in the target color range in the YCbCr color space by converting each pixel of the target pixel candidate in the input image into the YCbCr color space, So as to determine the target pixel.
  • the target pixel can be determined by whether at least one of the YCrCb color spaces exists in the target color range.
  • the color space used by the color space determination unit 5 for determination is not limited to the aforementioned HSV color space and YCrCb color space, and various color spaces such as RGB color space can be used.
  • the color space determination unit 5 can also set multiple target color ranges when determining the target pixel.
  • the color space determination unit 5 may also set a reference point in the target color range, set a color score corresponding to the distance from the reference point to the color point of each pixel, and set the color score to exceed a predetermined threshold.
  • the pixel serves as the target pixel.
  • each pixel of the area in which the circular parts in the areas DR3 and DR4 are enlarged corresponding to the size of the input image may become the target pixel candidate.
  • the reliability score at the time of determining the detection area of the area determination circuit 2 is used for the area score, all pixels in the target summary area or the determination small area become the same area score.
  • pixels corresponding to (background) other than the face part also become target pixel candidates.
  • a color score is calculated for each pixel of the target pixel candidate, and pixels of the background part other than the face part of each pixel of the target pixel candidate can be excluded from the target pixel by using the color score.
  • the color space determination unit 5 outputs to the image quality improvement processing circuit 3 the determination result of whether it is the target pixel or the color score information for each pixel of the target pixel candidate. It should be noted that the result of determining whether it is a target pixel can also be expressed as color score information as described above. Therefore, in the following description, the color score information is supplied to the image quality improvement processing circuit 3 Be explained.
  • the image quality improvement processing circuit 3 constituting the image processing circuit performs image quality improvement processing by performing predetermined image quality processing on the input image.
  • the image quality improvement processing circuit 3 may set the processing parameters of the image quality processing based on the color score information for each pixel.
  • the image quality improvement processing circuit 3 may use a pixel with a color score higher than a predetermined threshold as a target pixel, and set processing parameters suitable for sharpening processing on the target pixel, thereby performing the sharpening processing.
  • the image quality improvement processing circuit 3 can also set suitable reduction for pixels other than pixels with color scores higher than a predetermined threshold in the input image, or pixels with color scores below a predetermined threshold among the pixels of the target pixel candidate.
  • the processing parameters of the noise processing to implement noise reduction processing. Folding noise is prone to appear at the boundary between a target such as a textured human face and a relatively smooth background.
  • the image quality improvement processing circuit 3 can improve the image quality of the target object by removing such noise or sharpening processing.
  • the image quality improvement processing circuit 3 is not limited to sharpening processing and noise reduction processing, and various image processing such as super-resolution processing can also be performed.
  • super-resolution processing the processing parameters of each pixel can be changed in accordance with the color score.
  • the processing parameters may be set for each pixel based on the values of the area score and the color score. Processing parameters.
  • the processing parameters can be changed not only for each pixel, but also for each predetermined area.
  • Fig. 4 is a flowchart for explaining the operation of the embodiment.
  • Moving images and the like are input as input images to the reduction circuit 1, the area determination circuit 2, and the image quality improvement processing circuit 3.
  • the flowchart of FIG. 4 shows the processing for each frame of the input moving image, and each circuit in FIG. 1 executes the processing of FIG. 4 on a predetermined frame.
  • the reduction circuit 1 performs reduction processing in step S1 in FIG. 4.
  • the input image is converted into a reduced image through a prescribed reduction algorithm.
  • the reduced image is supplied to the target detection unit 4.
  • the target detection unit 4 uses machine learning technology to detect the target object (step S2). For example, the target detection unit 4 obtains a rectangular detection area as the image area of the target object. The detection result of the target detection unit 4 is supplied to the area determination circuit 2, and the area determination circuit 2 obtains the target summary area in which the detection area is enlarged to the position and size of the original input image (step S3).
  • the area determination circuit 2 obtains an area score for determining whether it is a candidate for a pixel constituting the target object for each pixel in the target summary area (step S4).
  • the area determination circuit 2 determines pixels whose area scores are greater than the threshold value as target pixel candidates (step S5).
  • the information of the target pixel candidate is supplied to the color space determination unit 5.
  • the color space determination unit 5 obtains a color score for each pixel of the target pixel candidate (step S6). For example, the color space determination unit 5 obtains a color score based on the relationship between the color of the target pixel candidate pixel and the target color range in a predetermined color space. That is, for example, the larger the color score, the closer the color of the pixel to the color of the target object in the color space. Therefore, by using the color score, it is possible to more accurately determine whether or not each pixel of the target pixel candidate is a target pixel.
  • the color space determination unit 5 outputs information on the color score of each pixel of the target pixel candidate to the image quality improvement processing circuit 3.
  • the image quality improvement processing circuit 3 sets, for example, processing parameters for image quality processing of the input image for each pixel based on the color score (step S7), and performs image quality improvement processing (step S8).
  • the image quality improvement processing circuit 3 sets processing parameters suitable for sharpening processing for target pixels whose color scores are higher than a predetermined threshold, and sets processing parameters suitable for noise reduction processing for pixels other than the target pixels, and performs image quality improvement processing. .
  • the image quality of target objects such as human faces can be improved.
  • the image quality improvement processing circuit 3 by setting different processing parameters for the target pixel and pixels other than the target pixel, the image quality of the pixel portion other than the target object is reduced. This can also improve the relative image quality of the target object. For example, for a predetermined target object, the image quality of other objects in the image can also be lowered. In this case, the visibility of the target object can be relatively improved.
  • the detector is used to determine the target object in the moving image, but also the color space of the detected target area is determined, so that the image quality improvement for the target and its surroundings can be controlled with high accuracy.
  • the target object is not particularly limited.
  • animals such as dogs and cats, cars, and balls can also be set.
  • the image quality of the golf ball can be improved in the moving image of the golf ball, and image quality improvement processing such as the dents can be clearly displayed.
  • each circuit (reduction circuit 1, area determination circuit 2, image quality improvement processing circuit 3, target detection unit 4, and color space determination unit 5) of the above-mentioned embodiment, the various parts constituting each circuit can be used as Each electronic circuit may be configured, or may be configured as a circuit module in an integrated circuit.
  • each circuit may also include one or more CPUs.
  • each circuit may be configured to read a program for executing the function of each part from a storage medium such as a memory, and perform an operation corresponding to the read program.
  • the present application is not limited to the above-mentioned embodiment, and various changes can be made in the implementation stage without departing from the scope of the spirit.
  • the above-mentioned embodiment includes inventions of various stages, and various inventions are extracted by appropriately combining a plurality of disclosed components. For example, even if some of the components are eliminated from all the components disclosed in the embodiment, as long as the problems described in the section of the problem to be solved by the invention can be solved and the effects described in the section of the effect of the invention can be obtained, the The structure whose constituent elements are eliminated can be extracted as an invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本申请的实施方式涉及图像处理装置及图像处理方法，进行基于机器学习的目标检测，并且利用颜色空间的信息，从而进行对于目标的有效的画质调节处理。根据实施方式，图像处理装置具备：缩小部，其将输入的图像进行缩小而输出缩小图像；目标检测部，其从上述缩小图像中检测预先确定的目标对象；区域判定部，其基于上述目标检测部的检测结果，对上述输入的图像中的包含上述目标对象在内的目标候选区域进行判定；颜色空间判定部，其基于与上述目标对象对应的颜色空间的信息，判定上述目标候选区域是否为与上述目标对象对应的区域；以及图像处理电路，其基于上述颜色空间判定部的判定结果，控制对于上述输入的图像的图像处理。

Description

图像处理装置及图像处理方法
本申请要求在2019年6月27日提交日本专利局、申请号为2019-120131、发明名称为“图像处理装置及图像处理方法”的日本专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请的实施方式涉及图像处理装置及图像处理方法。
背景技术
以往，例如利用超分辨率处理、锐化处理、降噪处理等各种图像处理技术来实现图像的画质提高。在进行这种画质提高处理的图像处理装置中，通过实施与图像中的目标对应的图像处理，从而实现更优异的画质提高。
例如，有时检测作为识别对象而重要的目标人物的面部，进行考虑检测出的面部区域的超分辨率处理、降噪处理等。此外，近年来，作为面部检测的方法，有时进行利用深度学习的处理。在该情况下，为了减少用于面部检测的运算量，还可以使用缩小图像进行面部区域的归纳处理。
然而，根据使用了缩小图像的面部区域的判定结果无法确定正确的面部区域，存在无法实现充分的改善画质的问题。
在先技术文献
专利文献
专利文献1：日本特开2019-40382号
发明内容
本申请实施方式的目的在于提供一种，通过进行基于机器学习的目标检测并且利用颜色空间的信息，从而能够进行对于目标的有效画质调节处理的图像处理装置及图像处理方法。
本申请实施方式所涉及的图像处理装置具备：缩小部，其将输入的图像进行缩小而输出缩小图像；目标检测部，其从上述缩小图像中检测预先确定的目标对象；区域判定部，其基于上述目标检测部的检测结果，对上述输入的图像中包含上述目标对象在内的目标候选区域进行判定；颜色空间判定部，其基于与上述目标对象对应的颜色空间的信息，判定上述目标候选区域是否为与上述目标对象对应的区域；以及图像处理电路，其基于上述颜色空间判定部的判定结果，控制对于上述输入的图像的图像处理。
附图说明
图1是表示本申请的一实施方式涉及的图像处理装置的框图；
图2是用于说明目标检测部4的处理的一例的说明图；
图3是用于说明目标检测部4的处理的一例的说明图；
图4是用于说明实施方式的动作的流程图。
附图标记说明
1…缩小电路，2…区域判定电路，3…画质改善处理电路，4…目标检测部，5…颜色空间判定部。
具体实施方式
下面，参照附图对本申请的实施方式详细地进行说明。
图1是表示本申请的一实施方式涉及的图像处理装置的框图。本实施方式通过检测器对动态图像中的目标进行判定，并且对检测出的目标区域进行颜色空间的判定，从而高精度地控制对于目标或其附近的画质改善处理的方法，所述检测器使用基于机器学习而得到的归纳模型。由此，能够使图像中的例如人的面部等目标的画质改善。
本实施方式的图像处理装置可以用于进行图像处理的各种装置。例如，可以将本实施方式的图像处理装置用于电视接收机、录像机等，使广播节目的图像中的各种目标画质改善，结果能够在图像整体上得到高画质的动态图 像。此外，例如还可以将本实施方式的图像处理装置用于监控摄像头、车载摄像头等，使拍摄到的动态图像中的各种目标画质改善，结果还可以提高人等目标的识别精度。
在图1中，输入图像提供给缩小电路1、区域判定电路2、以及画质改善处理电路3。输入图像是基于规定的帧率、规定的分辨率、规定的标准的动态图像。例如，可以是基于通过电视接收机等接收的广播信号的动态图像，也可以是从规定的相机系统得到的动态图像。
作为缩小部的缩小电路1对输入图像进行缩小处理。例如，缩小电路1可以采用公知的双线性法(Bilinear)、双三次法(Bicubic)等各种缩小算法，算法没有特别限定。缩小电路1从输入图像获取缩小图像。需要说明的是，缩小倍率取决于输入图像尺寸、目标检测部4的运算速度。缩小电路1将以规定的帧率生成的缩小图像依次向目标检测部4进行输出。
目标检测部4使用机器学习技术进行从被输入的缩小图像中对检测对象的目标(以下称为目标对象)进行检测的处理。需要说明的是，目标对象还可以是事先确定的目标。在目标检测部4中，构建用于目标对象检测的推论模型的规定的网络由硬件或软件构成。
关于目标检测部4的推论模型，通过将表示缩小图像中的目标对象的范围的信息作为标签附加到缩小图像而制作出的大量的训练数据提供给规定的网络而学习，从而得到该推论模型。该归纳模型针对缩小图像的输入，将表示目标对象的范围的信息与其可靠性的信息一起输出。需要说明的是，作为规定的网络，还可以采用DNN(深度神经网络)。此外，作为机器学习的方法，目标检测部4还可以利用深度神经网络以外的方法、例如哈尔特征(Haar-Like)等方法。
图2和图3是用于说明目标检测部4的处理的一例的说明图。图2和图3表示目标对象为人的面部的情况下的检测处理的例子。
图2和图3的缩小图像Pin表示输入到目标检测部4中的缩小图像。该缩小图像Pin中包含有人物O1、O2的图像，圆形表示作为目标对象的面部部分 的图像。在图2的例子中，目标检测部4通过归纳处理，如缩小图像Pout所示，将包含人物O1的面部部分在内的区域DR1和包含人物O2的面部部分在内的区域DR2作为目标对象的检测区域进行检测。例如，目标检测部4对面部部分进行检测，将以检测出的面部部分的中心的坐标为中心的规定尺寸的矩形区域作为检测区域。目标检测部4将关于区域DR1、DR2的信息作为目标对象的检测结果进行输出。
另一方面，图3表示按照通过网格对缩小图像Pin进行划分的小区域(以下称为判定小区域)来检测目标对象的范围的例子。在该情况下，关于构成目标检测部4的归纳模型，可以通过将按照每个判定小区域附加了表示是否为目标对象的标签的缩小图像作为训练数据的学习来获取该归纳模型。
因此，目标检测部4通过归纳处理，如缩小图像Pout所示，将区域DR3和区域DR4作为目标对象的检测区域进行检测，其中，所述区域DR3包括作为人物O1的面部部分而检测出的2个判定小区域，所述区域DR4包括作为人物O2的面部部分而检测出的4个判定小区域。目标检测部4将与区域DR3、DR4相关的信息作为目标对象的检测结果进行输出。
目标检测部4将与检测区域相关的信息输出到区域判定电路2。作为区域判定部的区域判定电路2在图2和图3的例子中的任一情况下，都将对缩小图像检测出的检测区域转换为与输入图像的尺寸相对应的位置和尺寸的区域(以下称为目标归纳区域)。
目标检测部4对该目标归纳区域的输入图像，求出认为构成目标对象的区域的候选(以下称为目标候选区域)。例如，目标检测部4对该目标归纳区域的输入图像，对于目标归纳区域内的每个像素，判定是否为目标候选区域内的像素、即构成目标对象的像素的候选(以下称为目标像素候选)。
例如，区域判定电路2还可以将检测区域的判定时的可靠性的得分作为用于判定目标归纳区域的各个像素是否为目标像素候选的得分(以下称为区域得分)。在该情况下，在图2的例子中，与区域DR1对应的目标归纳区域内的全部像素成为彼此相同区域得分，与区域DR2对应的目标归纳区域内的 全部像素成为彼此相同区域得分。此外，在图3的例子中，在分别与区域DR3、DR4的各个判定小区域对应的每个目标归纳区域，目标归纳区域内的全部像素成为彼此相同区域得分。
需要说明的是，区域判定电路2不仅可以利用检测区域的判定时的可靠性的得分，还可以利用其它信息来确定区域得分。区域判定电路2还可以将区域得分超过规定阈值的像素作为目标像素候选。
在本实施方式中，为了求出构成目标对象的像素即目标像素，将目标像素候选提供给颜色空间判定部5。需要说明的是，目标像素是使用对于目标对象的处理参数进行图像处理的像素。
颜色空间判定部5基于目标像素候选的像素是否保持有相当于目标对象的颜色空间的信息来判定目标像素。例如，在目标对象为人的面部的情况下，在目标像素候选的像素的颜色信息表示人的肤色(面部颜色)的情况下，能够判定为该像素保持有相当于目标对象的颜色空间的信息。
例如，颜色空间判定部5还可以将输入图像中的目标像素候选的各个像素转换为规定的颜色空间的信息，从而判定其颜色。例如，颜色空间判定部5将输入图像中的目标像素候选的各个像素转换为HSV颜色空间，按照每个像素判定像素的颜色是否存在于与HSV颜色空间内的目标对象的颜色对应的规定的范围(以下称为目标颜色范围)内，从而判定目标像素。此外，还可以通过HSV颜色空间内的色调(H)、彩度(S)和亮度(V)中的至少一个是否存在于目标颜色范围内来判定目标像素。
此外，例如，颜色空间判定部5还可以通过将输入图像中的目标像素候选的各个像素转换为YCbCr颜色空间，按照每个像素判定像素的颜色是否存在于YCbCr颜色空间内的目标颜色范围内，从而判定目标像素。此外，即使在该情况下，也可以通过YCrCb颜色空间中的至少一个是否存在于目标颜色范围内来判定目标像素。
需要说明的是，作为颜色空间判定部5在判定中使用的颜色空间，并不限定于上述的HSV颜色空间、YCrCb颜色空间，可以采用RGB颜色空间等 各种颜色空间。在将人的面部作为目标对象的情况下，根据人种等而目标颜色范围不同。因此，颜色空间判定部5在判定目标像素时还可以设定多个目标颜色范围。
此外，在上述说明中，说明了通过目标像素候选的各个像素的颜色是否存在于目标颜色范围内来判定是否为目标像素的例子。对此，颜色空间判定部5还可以在目标颜色范围内设置基准点，设定与从该基准点至各个像素的颜色的点为止的距离相对应的颜色得分，将颜色得分超过了规定阈值的像素作为目标像素。上述的通过是否存在于目标颜色范围内来判定是否为目标像素的例子，可以说是目标颜色范围内的颜色得分为最大值且目标颜色范围外的颜色得分为最小值的例子。
例如，在图3的例子中，根据区域得分的结果，将区域DR3、DR4中的圆形部分与输入图像的尺寸相对应地放大的区域的各个像素可以成为目标像素候选。但是，如上所述，在将区域判定电路2的检测区域的判定时的可靠性的得分用于区域得分的情况下，目标归纳区域内或判定小区域内的全部像素成为相同区域得分。其结果是，特别是在面部的轮廓部分，对于与除了面部部分以外(背景)对应的像素也成为目标像素候选。
在本实施方式中，对目标像素候选的各个像素求出颜色得分，关于目标像素候选的各个像素中除了面部部分以外的背景部分的像素，通过使用颜色得分，从而能够从目标像素中排除。
颜色空间判定部5按照目标像素候选的每个像素，向画质改善处理电路3输出是否为目标像素的判定结果、或者颜色得分的信息。需要说明的是，关于是否为目标像素的判定结果，如上所述，也可以作为颜色得分的信息而表现，因此在以下说明中，作为颜色得分的信息被供给到画质改善处理电路3的情况进行说明。
构成图像处理电路的画质改善处理电路3通过对输入图像进行规定的画质处理，从而实施画质改善处理。在本实施方式中，对于输入图像或输入图像中的目标像素候选，画质改善处理电路3也可以按照每个像素而基于颜色 得分的信息来设定画质处理的处理参数。
例如，画质改善处理电路3可以将颜色得分高于规定阈值的像素作为目标像素，对目标像素设定适合锐化处理的处理参数，从而实施锐化处理。此外，画质改善处理电路3还可以对在输入图像中除了颜色得分高于规定阈值的像素以外的像素、或者在目标像素候选的各个像素中颜色得分为规定阈值以下的像素，设定适合降噪处理的处理参数，从而实施降噪处理。在有纹理的人的面部等目标与比较平滑的背景的边界部分，容易出现折叠噪声。画质改善处理电路3通过去除这种噪声或锐化处理，从而能够使目标对象的画质改善。
需要说明的是，画质改善处理电路3并不限定于锐化处理、降噪处理，还可以进行各种图像处理、例如超分辨率处理等。在超分辨率处理中，可以将每个像素的处理参数与颜色得分相对应地进行变更。另外，虽然说明了只与区域得分大于规定阈值的目标像素候选的各个像素的颜色得分相对应地设定处理参数的例子，但也可以根据区域得分及颜色得分的值而按照每个像素设定处理参数。此外，处理参数不仅是按照每个像素，还可以按照每个规定的区域进行变更。
接着，参照图4对如上述方式构成的实施方式的操作进行说明。图4是用于说明实施方式的操作的流程图。
动态图像等作为输入图像输入到缩小电路1、区域判定电路2和画质改善处理电路3中。图4的流程图表示被输入的动态图像的每个帧的处理，图1的各个电路对规定的帧执行图4的各个处理。
缩小电路1在图4的步骤S1中进行缩小处理。输入图像通过规定的缩小算法转换为缩小图像。该缩小图像被供给到目标检测部4中。
目标检测部4利用机器学习技术检测目标对象(步骤S2)。例如，目标检测部4求出矩形的检测区域作为目标对象的图像区域。目标检测部4的检测结果被供给到区域判定电路2中，区域判定电路2求出将检测区域放大为原始输入图像的位置和尺寸的目标归纳区域(步骤S3)。
区域判定电路2按照目标归纳区域内的每个像素，求出用于判定是否为构成目标对象的像素的候选的区域得分(步骤S4)。区域判定电路2将区域得分大于阈值的像素确定为目标像素候选(步骤S5)。
目标像素候选的信息被供给到颜色空间判定部5中。颜色空间判定部5对目标像素候选的各个像素求出颜色得分(步骤S6)。例如，颜色空间判定部5基于在规定的颜色空间中目标像素候选的像素的颜色与目标颜色范围之间的关系，求出颜色得分。即，例如颜色得分为越大的值，认为其像素的颜色在颜色空间上是越接近目标对象的颜色的颜色。因此，通过使用颜色得分，从而能够更高精度地进行目标像素候选的各个像素是否为目标对象的像素的判定。
颜色空间判定部5将目标像素候选的每个像素的颜色得分的信息输出到画质改善处理电路3中。画质改善处理电路3根据颜色得分而例如将针对输入图像的画质处理的处理参数设定于每个像素(步骤S7)，实施画质改善处理(步骤S8)。
例如，画质改善处理电路3对于颜色得分高于规定阈值的目标像素设定适合锐化处理的处理参数，对于除了目标像素以外的像素设定适合降噪处理的处理参数，实施画质改善处理。由此，能够使人物面部等目标对象的画质改善。
需要说明的是，在本实施方式中，在画质改善处理电路3中，通过对目标像素和目标像素以外的像素设定彼此不同的处理参数，使目标对象以外的像素部分低画质化，从而还能够使目标对象相对画质改善。例如，对于预先确定的目标对象，还可以使图像中的其它目标的画质低画质化，在该情况下，能够相对提高目标对象的可视性。
如此在本实施方式中，不仅通过检测器对动态图像中的目标对象进行判定，还对检测出的目标的区域进行颜色空间的判定，从而能够高精度地控制对于目标、其周围的画质改善处理的方法，其中，所述检测器使用基于机器学习而得到的归纳模型。由此，能够使图像中的例如人物面部等目标画质改 善，能够提高动态图像中目标的可视性，并且还可以提高针对目标的识别精度。
需要说明的是，在上述实施方式中，虽然将人物面部作为目标对象为例进行了说明，但目标对象没有特别限定。例如，作为目标对象，还可以设定狗、猫等动物、汽车、球等。例如，在将高尔夫球设定为目标对象的情况下，在跟踪高尔夫球的动态图像中提高高尔夫球的画质，可以实现连凹痕都能清楚地显示等画质改善处理。
需要说明的是，在上述实施方式的各个电路(缩小电路1、区域判定电路2、画质改善处理电路3、目标检测部4及颜色空间判定部5)中，构成各个电路的各个部分可以作为各个电子电路而构成，或者还可以作为集成电路中的电路模块构成。此外，各个电路还可以具备1个以上的CPU而构成。此外，各个电路可以设置为从存储器等存储介质读取用于执行各个部分的功能的程序，并且进行与该读取的程序相对应的动作。
本申请并不限定于上述实施方式，在实施阶段中可以在不脱离其主旨的范围内进行各种各样的改变。此外，上述实施方式中包含各种阶段的发明，通过适当地组合被公开的多个构成要素来提取各种发明。例如，即使从实施方式中公开的全部构成要素中消除几个构成要素，只要能够解决发明所要解决的课题的栏中叙述的课题，能够得到发明的效果的栏中叙述的效果的情况下，该构成要素被消除的构成可以作为发明而提取。

Claims (7)

  1. 一种图像处理装置，包括：
    缩小部，用于将输入的图像进行缩小而输出缩小图像；
    目标检测部，用于从所述缩小图像中检测预先确定的目标对象；
    区域判定部，用于基于所述目标检测部的检测结果，对所述输入的图像中包含所述目标对象在内的目标候选区域进行判定；
    颜色空间判定部，用于基于与所述目标对象对应的颜色空间的信息，判定所述目标候选区域是否为与所述目标对象对应的区域；以及
    图像处理电路，用于基于所述颜色空间判定部的判定结果，控制对于所述输入的图像的图像处理。
  2. 根据权利要求1所述的图像处理装置，其中，
    所述目标检测部，还使用基于神经网络的归纳处理，从所述缩小图像中检测所述目标对象。
  3. 根据权利要求1所述的图像处理装置，其中，
    所述区域判定部，还用于按照所述输入的图像的每个像素来判定构成所述目标候选区域的目标像素候选。
  4. 根据权利要求3所述的图像处理装置，其中，
    所述颜色空间判定部，还用于对于所述目标像素候选，按照每个像素判定是否为构成所述目标对象的目标像素。
  5. 根据权利要求4所述的图像处理装置，其中，
    所述图像处理电路，还用于对所述目标像素和所述目标像素以外的像素实施使用不同处理参数的图像处理。
  6. 根据权利要求4所述的图像处理装置，其中，
    所述图像处理电路，还用于设定处理参数，该处理参数用于对所述目标像素实施锐化处理，并且对所述目标像素以外的像素实施降噪处理。
  7. 一种图像处理方法，包括：
    将输入的图像缩小而输出缩小图像，
    从所述缩小图像中检测预先确定的目标对象，
    基于所述目标对象的检测结果，对所述输入的图像中的包含所述目标对象在内的目标候选区域进行判定，
    基于与所述目标对象对应的颜色空间的信息，判定所述目标候选区域是否为与所述目标对象对应的区域，以及
    基于使用所述颜色空间的信息的判定结果，控制对于所述输入的图像的图像处理。
PCT/CN2020/098169 2019-06-27 2020-06-24 图像处理装置及图像处理方法 WO2020259603A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202080002690.8A CN112470165B (zh) 2019-06-27 2020-06-24 图像处理装置及图像处理方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019120131A JP2021005320A (ja) 2019-06-27 2019-06-27 画像処理装置及び画像処理方法
JP2019-120131 2019-06-27

Publications (1)

Publication Number Publication Date
WO2020259603A1 true WO2020259603A1 (zh) 2020-12-30

Family

ID=74059858

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/098169 WO2020259603A1 (zh) 2019-06-27 2020-06-24 图像处理装置及图像处理方法

Country Status (3)

Country Link
JP (1) JP2021005320A (zh)
CN (1) CN112470165B (zh)
WO (1) WO2020259603A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040151371A1 (en) * 2003-01-30 2004-08-05 Eastman Kodak Company Method for face orientation determination in digital color images
CN103473564A (zh) * 2013-09-29 2013-12-25 公安部第三研究所 一种基于敏感区域的正面人脸检测方法
CN104751136A (zh) * 2015-03-11 2015-07-01 西安理工大学 一种基于人脸识别的多相机视频事件回溯追踪方法
CN107909551A (zh) * 2017-10-30 2018-04-13 珠海市魅族科技有限公司 图像处理方法、装置、计算机装置及计算机可读存储介质
WO2018225061A1 (en) * 2017-06-04 2018-12-13 De-Identification Ltd. System and method for image de-identification

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2000076454A (ja) * 1998-08-31 2000-03-14 Minolta Co Ltd ３次元形状データ処理装置
JP2006011685A (ja) * 2004-06-24 2006-01-12 Noritsu Koki Co Ltd 写真画像処理方法及びその装置
JP2010152521A (ja) * 2008-12-24 2010-07-08 Toshiba Corp 画像立体化処理装置及び方法
JP4983962B2 (ja) * 2009-07-23 2012-07-25 カシオ計算機株式会社 画像処理装置、画像処理方法及び画像処理プログラム
JP5486678B2 (ja) * 2010-05-21 2014-05-07 シャープ株式会社 色判定装置、色判定方法、画像処理回路及びプログラム
JP2012235407A (ja) * 2011-05-09 2012-11-29 Sony Corp 画像処理装置および画像処理方法
JP5980555B2 (ja) * 2012-04-23 2016-08-31 オリンパス株式会社 画像処理装置、画像処理装置の作動方法、及び画像処理プログラム

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040151371A1 (en) * 2003-01-30 2004-08-05 Eastman Kodak Company Method for face orientation determination in digital color images
CN103473564A (zh) * 2013-09-29 2013-12-25 公安部第三研究所 一种基于敏感区域的正面人脸检测方法
CN104751136A (zh) * 2015-03-11 2015-07-01 西安理工大学 一种基于人脸识别的多相机视频事件回溯追踪方法
WO2018225061A1 (en) * 2017-06-04 2018-12-13 De-Identification Ltd. System and method for image de-identification
CN107909551A (zh) * 2017-10-30 2018-04-13 珠海市魅族科技有限公司 图像处理方法、装置、计算机装置及计算机可读存储介质

Also Published As

Publication number Publication date
JP2021005320A (ja) 2021-01-14
CN112470165B (zh) 2023-04-04
CN112470165A (zh) 2021-03-09

Similar Documents

Publication Publication Date Title
US10684681B2 (en) Neural network image processing apparatus
US20190130169A1 (en) Image processing method and device, readable storage medium and electronic device
US10872262B2 (en) Information processing apparatus and information processing method for detecting position of object
JP5567853B2 (ja) 画像認識装置および方法
US10506174B2 (en) Information processing apparatus and method for identifying objects and instructing a capturing apparatus, and storage medium for performing the processes
US8463049B2 (en) Image processing apparatus and image processing method
JP6688277B2 (ja) プログラム、学習処理方法、学習モデル、データ構造、学習装置、および物体認識装置
US7995807B2 (en) Automatic trimming method, apparatus and program
EP3168810A1 (en) Image generating method and apparatus
US10529103B2 (en) Image processing apparatus and method for collating a plurality of images
US20230360435A1 (en) Facial authentication device, facial authentication method, and program recording medium
WO2009095168A1 (en) Detecting facial expressions in digital images
US7460705B2 (en) Head-top detecting method, head-top detecting system and a head-top detecting program for a human face
JP2021503139A (ja) 画像処理装置、画像処理方法および画像処理プログラム
JP2009059073A (ja) 撮影装置、撮影方法、人物認識装置および人物認識方法
CN113822927A (zh) 一种适用弱质量图像的人脸检测方法、装置、介质及设备
JP2007026308A (ja) 画像処理方法、画像処理装置
WO2020259603A1 (zh) 图像处理装置及图像处理方法
JP2018063675A (ja) 画像処理装置および制御方法
JP5539565B2 (ja) 撮像装置及び被写体追跡方法
US11763596B2 (en) Image capturing support apparatus, image capturing support method, and computer-readable recording medium
JP4789526B2 (ja) 画像処理装置、画像処理方法
US20220207261A1 (en) Method and apparatus for detecting associated objects
US20160364604A1 (en) Subject tracking apparatus, control method, image processing apparatus, and image pickup apparatus
JP7338174B2 (ja) 物体検出装置および物体検出方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20832539

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20832539

Country of ref document: EP

Kind code of ref document: A1