WO2018068420A1 - 图像处理方法和装置 - Google Patents
图像处理方法和装置 Download PDFInfo
- Publication number
- WO2018068420A1 WO2018068420A1 PCT/CN2016/113270 CN2016113270W WO2018068420A1 WO 2018068420 A1 WO2018068420 A1 WO 2018068420A1 CN 2016113270 W CN2016113270 W CN 2016113270W WO 2018068420 A1 WO2018068420 A1 WO 2018068420A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- area
- pixel
- depth
- depth value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Definitions
- Embodiments of the present invention relate to the field of image processing technologies, and in particular, to an image processing method and apparatus.
- the focus of beautification is often the foreground part of the image.
- the main beautification object is the face in the foreground.
- Mito technology solutions There are two main types of existing Mito technology solutions: First, the entire image is processed. The problem with this type of method is that it does not distinguish between regions and does not distinguish the key points from the whole map. It not only wastes a lot of unnecessary calculation time, but also has low efficiency, and can not satisfy the user's partial beautification, highlight the image focus, and pass different beautifications. The effect enhances the individual needs of the image hierarchy and the like.
- the target area detection method such as face detection and skin color detection is used to beautify based on the detection result of the target area.
- the method reduces the calculation time, the result of image processing often depends on the accuracy of target area detection, and the target area detection methods such as face detection and skin color detection have limited target area and omission result due to limited robustness. Not quite satisfactory.
- the invention provides an image processing method and device, which can solve the problem that image processing cannot be divided into regions or objects. High-precision image segmentation and efficient image processing are achieved by inaccurate detection of standard areas.
- an embodiment of the present invention provides an image processing method, where the method includes:
- the processed area image is fused with the RGB image.
- an embodiment of the present invention further provides an image processing apparatus, where the apparatus includes:
- An original image acquisition module configured to acquire a depth image and an RGB image in the same scene
- An area image processing module configured to divide the RGB image according to a depth value in the depth image to obtain at least two area images, and process at least one of the area images;
- An image fusion module is configured to fuse the processed area image with the RGB image.
- the RGB image is segmented by the depth value of the depth image in the same scene, and the precision is higher than that of the target region detected by using the RGB image alone.
- the fusion of the processed area image and the original RGB image can effectively reduce the distortion after image processing, preserve the integrity of the image information, and at the same time take into consideration the processing efficiency and the fusion effect.
- the method for processing the entire image is relatively indistinguishable, and by dividing the RGB image into at least two area images and processing only the area image selected by the user, the unnecessary calculation time can be greatly reduced.
- Real-time calculation; relative to the method based on specific target area detection, directly using depth information to segment RGB images is more accurate and robust.
- FIG. 1 is a schematic flowchart of an image processing method according to Embodiment 1 of the present invention.
- FIG. 2A is a schematic flowchart of an image processing method according to Embodiment 2 of the present invention.
- FIG. 2B is a schematic diagram of splitting an RGB image into two area images according to Embodiment 2 of the present invention.
- FIG. 3 is a schematic structural diagram of an image processing apparatus according to Embodiment 3 of the present invention.
- FIG. 1 is a flowchart of an image processing method according to Embodiment 1 of the present invention.
- the method may be implemented by an image processing apparatus, and the apparatus may be implemented by hardware and/or software, and is generally independent.
- the configuration of the embodiment is implemented in the terminal.
- the RGB image can be understood as a color image, and the color of the RGB image is changed by three reference color channels of red (Red, R), green (Green), blue (Blue, B) and their mutual Superimposed to get a variety of colors.
- the depth image can represent the color resolution of the image by the depth of the image, ie the number of bits used to store each pixel.
- the size of the pixel values in the depth image reflects the depth of the depth of field. In this embodiment, by acquiring the depth image and the RGB image in the same scene, more and more scene information can be acquired, so that more accurate image processing is performed by the image data.
- the acquired image is related to factors such as shooting distance, shooting angle, shooting environment, and shooting time. Therefore, the depth image and the RGB image in the same scene may be obtained by acquiring the depth image and the RGB image of the same scene at the same time.
- the depth image may be acquired by a depth camera, and the RGB image may be acquired by an ordinary camera; of course, the depth image and the RGB image may also be acquired by the same camera.
- dividing the original RGB image according to the depth value in the depth image to obtain the at least two area images may be: determining a second pixel value of the original RGB image according to the first pixel value of the depth image, and further dividing the original RGB according to the second pixel value.
- the image gets at least two area images.
- the segmented region image may also be labeled. For example, each area image is labeled based on the pixel value of each pixel of the area image. Alternatively, the pixel values of the pixels of the same image area may be labeled with the same value.
- processing at least one area image may be processing one, two or more area images.
- the user can select the area to be processed according to the actual segmentation situation and his own individual needs. Of course, it is also possible to process the corresponding area image according to the default setting.
- the original RGB image may be segmented according to the depth value in the depth image to obtain two regional images, wherein the pixel values of each pixel of the regional image may be represented by 1 or 0 respectively; wherein 1 represents the foreground region; Background area.
- the user can process the foreground and/or background regions based on pixel values of 1 or 0, as well as their own personalization needs. For example, a foreground area with a pixel value of 1 can be set by default.
- the processed area image may be for each area image of the full image or for the foreground area, in order to simultaneously calculate the calculation efficiency and the fusion effect, the processed area image and the unprocessed area image may be selected. Fusion. This technical solution is particularly suitable for the case of processing only individual region images after RGB image segmentation.
- Image fusion can include pixel level fusion, feature level fusion, and decision level fusion.
- spatial domain algorithm and transform domain algorithm in pixel level fusion there are multiple fusion rule methods in spatial domain algorithm, such as logic filtering method, gray weighted average method, contrast modulation method, etc.; pyramid domain decomposition fusion in transform domain algorithm Method, wavelet transform method, etc.
- the merging of the processed area image and the RGB image may be performed by combining pixel values of the processed area image with pixel values of the RGB image to preserve the original as much as possible. data.
- the RGB image is segmented by the depth value of the depth image in the same scene, and the precision is higher than that of the target region detected by using the RGB image alone. Moreover, it is possible to perform the division of the RGB image to obtain at least one of the at least two area images. The reason is that the area to be processed can be selected according to the needs of the user, and the same or different processing can be performed on different areas to meet the personalized needs of the user. The fusion of the processed area image and the original RGB image can effectively reduce the distortion after image processing, preserve the integrity of the image information, and at the same time take into consideration the processing efficiency and the fusion effect.
- the method for processing the entire image is relatively indistinguishable, and by dividing the RGB image into at least two area images and processing only the area image selected by the user, the unnecessary calculation time can be greatly reduced.
- Real-time calculation; relative to the method based on specific target area detection, directly using depth information to segment RGB images is more accurate and robust.
- the RGB image is divided according to the depth value in the depth image to obtain at least two area images
- processing at least one of the area images includes: Determining, according to the first depth value of each pixel point of the depth image, a second depth value corresponding to each pixel point of the RGB image; and dividing the RGB image according to the second depth value and a preset rule to obtain at least two The area image; processing at least one of the area images corresponding to the preset processing model.
- the method further includes: calculating the depth image before determining the second depth value corresponding to each pixel of the RGB image according to the first depth value of each pixel of the depth image. a mapping matrix of a first coordinate of each pixel and a second coordinate of each pixel of the RGB image; determining according to the second coordinate, the mapping matrix, and a first depth value of each pixel of the depth image a second depth value corresponding to each pixel point of the RGB image.
- the merging the processed area image with the RGB image may further include: performing Gaussian smoothing on the processed area image to obtain a smooth image of each pixel of the area image. a prime value; calculating a fused pixel value of the fused image according to the smoothed pixel value and a pixel value of each pixel of the RGB image; and outputting the fused image according to the fused pixel value.
- the method in this embodiment may include:
- the mapping matrix of the first coordinate and the second coordinate may be calculated by using a method of the prior art, for example, according to parameters of a camera that acquires a depth image and an RGB image, or may acquire a depth image and an RGB image.
- the part in which the features overlap, the correspondence between the depth image and the features of the RGB image is obtained based on the feature overlapping portion, and the mapping matrix is calculated according to the correspondence between the features.
- the pixel values of the depth image corresponding to each pixel point of the RGB image may be determined according to the second coordinate and the mapping matrix; and the first depth value of each pixel point of the acquired depth image is used as the pixel point of the depth image.
- a second depth value corresponding to each pixel of the corresponding RGB image may include: multiplying the coordinates of each pixel point of the RGB image by the mapping matrix. The coordinates of each pixel point of the corresponding depth image. The depth image is aligned with the RGB image, and the depth value of each pixel of the RGB image is obtained by the depth value of each pixel of the depth image.
- the preset rule may be: setting a depth threshold, and dividing the RGB image to obtain at least two area images according to the comparison result of the second depth value and the depth threshold. Specifically, according to the second depth value of each pixel point in the RGB image, and the preset depth threshold (or depth range), each pixel point in the RGB image is traversed, and the second depth value and the depth threshold of each pixel point are Or the depth range is compared, and the RGB image is divided into two area images according to the comparison result.
- the depth threshold can be a specific point value or a range.
- the specific numerical value or numerical interval of the depth threshold may be set by the user according to the actual image of the acquired RGB image, which is not limited herein.
- the segmentation of the original RGB image according to the second depth value and the preset rule to obtain the at least two region images may specifically determine whether the second depth value is greater than a preset depth threshold, and if so, then The position of the pixel corresponding to the depth value is used as the foreground area. If not, the position of the pixel corresponding to the second depth value is used as the background area; and the original RGB image is divided according to the foreground area and the background area to obtain two area images.
- the position of the pixel corresponding to the second depth value within the depth threshold range may be acquired, and as the foreground region, the set of positions of the remaining pixels is used as the background region.
- the depth threshold range may be set according to actual conditions, and then judged. Whether the second depth value of each pixel in the RGB image is within a preset depth threshold range, and if so, the pixel point is used as the foreground area, that is, the pixel part of the avatar part, if not within the preset depth threshold range, Then the pixel is used as the pixel of the background area.
- the pixel values of each pixel in the foreground area may be marked as 1 (black area in FIG.
- the set of the remaining pixels is used as the background area, and the pixel values of each pixel in the background area may be marked as 0 (FIG. 2B) Medium grid area). Further, the foreground area and/or the background area may be respectively determined according to the pixel values of the respective pixel points. Process it. It should be noted that the grid in FIG. 2B is only used to indicate that the pixel values of the background area are all marked as 0, and the grid area does not exist.
- the obtained image of the area to be processed may be affected by the noise of the depth image to present some loop-like areas, and these areas may be filled by the expansion etching operation to obtain the final area image.
- the at least two area images include two, three and more area images.
- the advantage of such setting can realize the partition processing or partial processing of the image, which can improve the efficiency of image processing and enrich the effect of image processing.
- the preset processing model may include a model for enhancing, restoring, segmenting, extracting features, removing noise, and the like, or two or more processing methods and techniques.
- the preset processing model may include a beautification processing algorithm.
- a bilateral filtering beautification processing algorithm may be used to implement a real-time dermabrasion image effect.
- processing at least one area image corresponding to the preset processing model includes processing one, two or more area images. Further, the processing models of the image of each region may be the same or different. Specifically, at least one area image may be acquired according to a selected instruction input by the user; and the selected at least one area image is processed based on the preset processing model.
- the size of the divided RGB image may be consistent with the size of the original RGB image, and the pixel points of the segmented RGB image are in one-to-one correspondence with the pixel points of the original RGB image, and further, after the segmentation
- the pixel value of each pixel in the RGB image can be only 1 or 0. Indicates whether the position of the RGB image corresponding to the pixel is the area image to be processed, respectively. As above, if divided into two area images, 1 can be used to indicate the foreground area; 0 is the background area.
- Gaussian smoothing is performed on the segmented image, and the pixel value a(x, y) of the segmented image thus obtained may take an arbitrary value between 0-1. This operation can effectively remove noise in the image and ensure image quality.
- the smoothed pixel value and the pixel value of each pixel of the RGB image may be weighted and summed to calculate a fused pixel value of the fused image. For example, suppose that the smoothed pixel value of the processed area image is I B (x, y), and the pixel value of each pixel of the original RGB image is I 0 (x, y).
- a(x, y) represents the weight of the smoothed pixel value. It can be understood that the specific value of a(x, y) can be set according to the actual situation, and can be a fixed value or calculated by other parameters, which is not limited herein.
- the technical solution of the embodiment determines the second depth value of each pixel of the RGB image by using the first depth value of each pixel of the depth image in the same scene, and can acquire more scene information by using the depth image and the RGB image.
- the accuracy of the region image segmentation can be improved; and then the RGB image is segmented according to the second depth value and the preset rule, and at least one region image corresponding to the preset processing model is processed, Dividing the image into at least one area image according to its own needs, and selecting one or more area images to be processed in at least one area image, and then processing the corresponding area image based on the preset processing model, not only enabling the user to
- the local processing of the image is realized according to the requirements of the user, and the user can meet the individualized requirements for different processing of the image of different regions; finally, the pixel value of the processed region image is merged with the pixel value of the original RGB image to generate the final fusion.
- the image is fused with pixel values, and the image
- FIG. 3 is a schematic structural diagram of an image processing apparatus according to Embodiment 3 of the present invention.
- the device can be implemented by means of hardware and/or software, and can generally implement the method of the embodiment in the terminal independently.
- the image processing apparatus specifically includes an original image acquiring module 310, an area image processing module 320, and an image fusion module 330.
- the original image obtaining module 310 is configured to acquire the depth image and the RGB image in the same scene, and the area image processing module 320 is configured to divide the RGB image according to the depth value in the depth image to obtain at least two regional images. Processing at least one of the area images; the image fusion module 330 is configured to fuse the processed area image with the RGB image.
- the area image processing module 320 may include: a depth value determining sub-module, an area image generating sub-module, and an area image processing sub-module.
- the depth value determining sub-module is configured to determine, according to the first depth value of each pixel point of the depth image, a second depth value corresponding to each pixel point of the RGB image; the area image generating sub-module is configured to be used according to the The second depth value and the preset rule divide the RGB image to obtain at least two area images; the area image processing sub-module is configured to process at least one of the area images corresponding to the preset processing model.
- the depth value determining submodule may further include a mapping matrix calculating unit and a second depth value calculating unit.
- the mapping matrix calculation unit is configured to calculate a mapping matrix of a first coordinate of each pixel of the depth image and a second coordinate of each pixel of the RGB image; a second depth value calculating unit, configured to And determining, by the second coordinate, the mapping matrix, and the first depth value of each pixel of the depth image, a second depth value corresponding to each pixel of the RGB image.
- the area image generation sub-module is specifically configured to: determine whether the second depth value is greater than a preset depth threshold, and if yes, the pixel point corresponding to the second depth value The position is the foreground area, and if not, the position of the pixel corresponding to the second depth value is used as the background area; and the RGB image is divided according to the foreground area and the background area to obtain two area images.
- the pixel values of the pixels of the area image are respectively represented by 1 or 0; wherein 1 represents the foreground area; 0 represents the background area.
- the image fusion module 330 is specifically configured to: perform Gaussian smoothing processing on the processed region image, and acquire smooth pixel values of each pixel of the region image; according to the smoothed pixel value and a pixel value of each pixel of the RGB image, calculating a fused pixel value of the fused image; and outputting the fused image according to the fused pixel value.
- the preset processing model may include a beautification processing algorithm.
- a bilateral filtering beautification processing algorithm may be used to implement a real-time microdermabrasion image effect.
- the embodiment further provides a terminal, which includes the image processing apparatus according to any embodiment of the present invention.
- the terminal may include a device having a photographing function such as a mobile phone, a tablet computer, a smart watch, and a camera.
- the image processing apparatus and the mobile terminal provided in the above embodiments may perform any embodiment of the present invention.
- the provided image processing method has the corresponding functional modules and beneficial effects for performing the method.
Landscapes
- Image Processing (AREA)
Abstract
公开了一种图像处理方法和装置,涉及图像处理技术领域。其中,该图像处理方法包括:获取同一场景下的深度图像和RGB图像;根据所述深度图像中的深度值分割所述RGB图像得到至少两个区域图像,处理至少一个所述区域图像;将处理后的区域图像与所述RGB图像进行融合。上述技术方案通过同一场景下的深度图像的深度值对RGB图像进行分割,相比较于单独采用RGB图像对目标区域进行检测,精度更高。而且,对分割RGB图像得到至少两个区域图像中的至少一个区域图像进行处理,能够根据自身需求选择所要处理的区域,且能够分别对不同的区域进行不同的处理,满足用户的个性化需求。将处理后的区域图像与原始RGB图像进行融合,同时兼顾处理效率与融合结果。
Description
本发明实施例涉及图像处理技术领域,尤其涉及一种图像处理方法和装置。
随着移动终端的普及,越来越多的用户习惯于借助移动终端的拍照功能,来记录生活,留住回忆。为了使得图片根据用户的偏好度进行显示,目前已经出现了大量的美化图像软件,或配置有美化图像功能的终端。
在许多图像美化应用场景中,美化的重点对象往往是图像中的前景部分。例如,如人脸美化时,主要的美化对象为前景中的人脸。目前,现有的美图技术方案主要有两类:第一,对整张图像进行处理。这类方法的问题在于,不区分区域,不区分重点地对全图进行处理,不仅浪费大量不必要的计算时间,效率较低,且不能满足用户局部美化,突出图像重点,以及通过不同的美化效果提升图像层次等的个性化需求。第二,通过人脸检测、肤色检测等目标区域检测方法,基于对目标区域的检测结果进行美化。该方法虽然减少了计算时间,但图像处理的结果往往依赖目标区域检测的精度,而诸如人脸检测、肤色检测等目标区域检测方法,由于鲁棒性有限,时有目标区域疏漏,导致美化结果不尽如人意。
发明内容
本发明提供一种图像处理方法和装置,以解决图像处理不能分区域或者目
标区域检测不精确等问题,实现高精度的图像分区及高效率的图像处理。
第一方面,本发明实施例提供了一种图像处理方法,该方法包括:
获取同一场景下的深度图像和RGB图像;
根据所述深度图像中的深度值分割所述RGB图像得到至少两个区域图像,处理至少一个所述区域图像;
将处理后的区域图像与所述RGB图像进行融合。
第二方面,本发明实施例还提供了一种图像处理装置,该装置包括:
原始图像获取模块,用于获取同一场景下的深度图像和RGB图像;
区域图像处理模块,用于根据所述深度图像中的深度值分割所述RGB图像得到至少两个区域图像,处理至少一个所述区域图像;
图像融合模块,用于将处理后的区域图像与所述RGB图像进行融合。
本发明的技术方案,通过同一场景下的深度图像的深度值对RGB图像进行分割,相比较于单独采用RGB图像对目标区域进行检测,精度更高。而且,能够对分割RGB图像得到至少两个区域图像中的至少一个所述区域图像进行处理,即能够根据自身需求选择所要处理的区域,且能够实现对不同的区域进行相同或不同的处理,满足用户的个性化需求。将处理后的区域图像与原始RGB图像进行融合,能够有效减少图像处理后的失真,保存图像信息的完整性,且同时兼顾处理效率与融合效果。上述技术方案,相对不区分区域、对整张图进行处理的方法来说,通过将RGB图像分割成至少两个区域图像,仅处理用户选择的区域图像,可以大量减少不必要的计算时间,实现实时计算;相对基于特定目标区域检测的方法来说,直接利用深度信息对RGB图像进行分割,精确性更高,鲁棒性更强。
图1为本发明实施例一所提供的一种图像处理方法的流程示意图;
图2A为本发明实施例二所提供的一种图像处理方法的流程示意图;
图2B为基于本发明实施例二所提供的分割RGB图像为两个区域图像的示意图;
图3为本发明实施例三所提供的一种图像处理装置的结构示意图。
下面结合附图并通过具体实施方式来进一步说明本发明的技术方案。可以理解的是,此处所描述的具体实施例仅仅用于解释本发明,而非对本发明的限定。另外还需要说明的是,为了便于描述,附图中仅示出了与本发明相关的部分而非全部结构。
在更加详细地讨论示例性实施例之前应当提到的是,一些示例性实施例被描述成作为流程图描绘的处理或方法。虽然流程图将各步骤描述成顺序的处理,但是其中的许多步骤可以被并行地、并发地或者同时实施。此外,各步骤的顺序可以被重新安排。当其操作完成时所述处理可以被终止,但是还可以具有未包括在附图中的附加步骤。所述处理可以对应于方法、函数、规程、子例程、子程序等等。
实施例一
图1为本发明实施例一提供的一种图像处理方法的流程图,该方法可以由图像处理装置来执行,该装置可通过硬件和/或软件的方式实现,并一般可独立
的配置在终端中实现本实施例的方法。
本实施例的方法具体包括:
S110、获取同一场景下的深度图像和RGB图像。
其中,RGB图像可以理解为彩色图像,RGB图像的色彩是通过对红(Red,R)、绿(Green,G)、蓝(Blue,B)三个基准颜色通道的变化以及它们相互之间的叠加来得到各式各样的颜色。深度图像可以通过图像深度,即存储每个像素所用的位数,来表征图像的色彩分辨率,深度图像中的像素值的大小反映了景深的远近。在本实施例中,获取同一场景下的深度图像和RGB图像,能够获取更多更丰富的场景信息,以便通过图像数据进行更精准地图像处理。
一般地,获取到的图像与拍摄距离、拍摄角度、拍摄环境与拍摄时间等因素有关。因此,获取同一场景下的深度图像和RGB图像具体可以是,获取同一时刻下同一场景的深度图像和RGB图像。具体地,深度图像可由深度摄像头获取,RGB图像可由普通摄像头获取;当然,也可以通过同一摄像头获取深度图像和RGB图像。
S120、根据所述深度图像中的深度值分割所述RGB图像得到至少两个区域图像,处理至少一个所述区域图像。
具体地,根据深度图像中的深度值分割原始RGB图像得到至少两个区域图像可以是,根据深度图像的第一像素值确定原始RGB图像的第二像素值,进而根据第二像素值分割原始RGB图像得到至少两个区域图像。进一步地,还可以对分割出的区域图像进行标注。例如,基于区域图像的各像素点的像素值对各区域图像进行标注。可选是,同一图像区域的各像素点的像素值可以相同的数值标注。
在本技术方案中,处理至少一个区域图像可以是处理一个、两个或者更多个区域图像。用户可以根据实际的分割情况及自身的个性化需求选择所要处理的区域。当然,也可以是根据默认设置处理对应的区域图像。
示例性地,可以根据深度图像中的深度值分割原始RGB图像得到两个区域图像,其中,区域图像的各像素点的像素值可分别用1或者0表示;其中,1表示前景区域;0表示背景区域。用户可以根据像素值为1或0,以及自身个性化需求处理前景区域和/或背景区域。例如,可以默认设置处理像素值为1的前景区域。
S130、将处理后的区域图像与所述RGB图像进行融合。
由于处理后的区域图像,可能针对全图的各个区域图像,也可能针对前景区域,因此为了同时兼顾计算效率与融合效果,可选是将处理后的区域图像与未经过任何处理的区域图像进行融合。该技术方案尤其适用于仅处理RGB图像分割后的个别区域图像的情况。
图像融合可包括像素级融合、特征级融合及决策级融合等。其中,像素级融合中有空间域算法和变换域算法,空间域算法中有多种融合规则方法,如逻辑滤波法,灰度加权平均法,对比调制法等;变换域算法中有金字塔分解融合法、小波变换法等。在本实施例中,将处理后的区域图像与所述RGB图像进行融合具体可以是,基于处理后的区域图像的像素值与所述RGB图像的像素值进行融合,以尽可能多得保存原始数据。
本发明的技术方案,通过同一场景下的深度图像的深度值对RGB图像进行分割,相比较于单独采用RGB图像对目标区域进行检测,精度更高。而且,能够对分割RGB图像得到至少两个区域图像中的至少一个所述区域图像进行处
理,即能够根据自身需求选择所要处理的区域,且能够实现对不同的区域进行相同或不同的处理,满足用户的个性化需求。将处理后的区域图像与原始RGB图像进行融合,能够有效减少图像处理后的失真,保存图像信息的完整性,且同时兼顾处理效率与融合效果。上述技术方案,相对不区分区域、对整张图进行处理的方法来说,通过将RGB图像分割成至少两个区域图像,仅处理用户选择的区域图像,可以大量减少不必要的计算时间,实现实时计算;相对基于特定目标区域检测的方法来说,直接利用深度信息对RGB图像进行分割,精确性更高,鲁棒性更强。
实施例二
图2A为本发明实施例二提供的一种图像处理方法的流程图。如图2A所示,本实施例在上述实施例的基础上,可选是根据所述深度图像中的深度值分割所述RGB图像得到至少两个区域图像,处理至少一个所述区域图像包括:根据所述深度图像的各像素点的第一深度值,确定所述RGB图像各像素点对应的第二深度值;根据所述第二深度值和预设的规则分割所述RGB图像得到至少两个区域图像;处理预设的处理模型对应的至少一个所述区域图像。
在此基础上,进一步地,在根据所述深度图像的各像素点的第一深度值,确定所述RGB图像各像素点对应的第二深度值之前,还可以包括:将计算所述深度图像的各像素点的第一坐标与所述RGB图像各像素点的第二坐标的映射矩阵;根据所述第二坐标、所述映射矩阵以及所述深度图像各像素点的第一深度值,确定所述RGB图像各像素点对应的第二深度值。
此外,将处理后的区域图像与所述RGB图像进行融合可选是还包括:对处理后的所述区域图像进行高斯平滑处理,获取所述区域图像各像素点的平滑像
素值;根据所述平滑像素值与所述RGB图像各像素点的像素值,计算融合图像的融合像素值;根据所述融合像素值输出融合图像。
具体地,本实施例的方法可以包括:
S201、获取同一场景下的深度图像和RGB图像。
S202、计算所述深度图像的各像素点的第一坐标与所述RGB图像各像素点的第二坐标的映射矩阵。
在本实施例中,第一坐标与第二坐标的映射矩阵可以采用现有技术的方法进行计算,例如根据获取深度图像和RGB图像的摄像头的参数进行计算,也可以是获取深度图像与RGB图像中特征重合的部分,基于特征重合部分获取深度图像与RGB图像的特征之间的对应关系,进而根据特征之间的对应关系计算出映射矩阵。
S203、根据所述第二坐标、所述映射矩阵以及所述深度图像各像素点的第一深度值,确定所述RGB图像各像素点对应的第二深度值。
具体地,可以是根据第二坐标和映射矩阵,确定与RGB图像各像素点对应的深度图像各像素点;将获取到的深度图像各像素点的第一深度值,作为与深度图像各像素点对应的RGB图像的各像素点所对应的第二深度值。示例性地,根据RGB图像各像素点的第一坐标和映射矩阵,确定与RGB图像各像素点对应的深度图像各像素点具体可以包括:将RGB图像各像素点的坐标与映射矩阵相乘获得对应的深度图像各像素点的坐标。即将深度图像与RGB图像对齐,通过深度图像各像素点的深度值获取RGB图像各像素点的深度值。
S204、根据所述第二深度值和预设的规则分割所述RGB图像得到至少两个区域图像。
其中,预设的规则可以是设定深度阈值,根据第二深度值与深度阈值的比对结果,分割RGB图像得到至少两个区域图像。具体可以根据RGB图像中的各像素点的第二深度值,以及预设的深度阈值(或深度范围),遍历RGB图像中的各像素点,将各像素点的第二深度值与深度阈值(或深度范围)进行比对,根据比对结果将RGB图像分割为两个区域图像。深度阈值可以是具体的一个点值,也可以是一个范围。深度阈值的具体数值或者数值区间,用户可以结合获取到的RGB图像的图像信息,根据实际需求进行设置,在此并不做限定。
在本实施例中,根据第二深度值和预设的规则分割原始RGB图像得到至少两个区域图像具体可以是,判断第二深度值是否大于预设的深度阈值,若是,则将与第二深度值对应的像素点的位置作为前景区域,若否,则将与第二深度值对应的像素点的位置作为背景区域;根据前景区域和背景区域,分割原始RGB图像得到两个区域图像。或者,也可以是获取位于深度阈值范围内的第二深度值对应的像素点的位置,作为前景区域,其余各像素点的位置的集合作为背景区域。
本方案中具体的以RGB图像分割为两个区域图像为例,如图2B所示,若要将头像部分作为前景区域,其余部分作为背景区域,可以根据实际情况设定深度阈值范围,进而判断RGB图像中的各像素点的第二深度值是否处于预设的深度阈值范围内,若是,则将该像素点作为前景区域即头像部分的像素点,若未处于预设的深度阈值范围内,则将该像素点作为背景区域的像素点。其中,前景区域各像素点的像素值可以均标记为1(图2B中黑色区域);将其余各像素点的集合作为背景区域,背景区域各像素点的像素值可以均标记为0(图2B中网格区域)。进而,可以根据各像素点的像素值分别对前景区域和/或背景区域
进行处理。需要说明的是,图2B中的网格仅仅用于表示背景区域的像素值均标记为0,不表示背景区域真实存在网格。
在实际的操作过程中,获得的所要处理的区域图像可能会受到深度图像的噪声的影响而呈现一些漏洞状的区域,此时可以通过膨胀腐蚀操作填补这些区域,得到最终的区域图像。
可以理解的是,至少两个区域图像包括两个、三个及更多个区域图像。这样设置的好处,可以实现对图像的分区处理或局部处理,能够提高图像处理的效率,丰富图像处理的效果。
S205、处理预设的处理模型对应的至少一个所述区域图像。
其中,预设的处理模型可以包括对图像进行增强、复原、分割、提取特征、去除噪声等一种、两种或多种处理方法和技术的模型。在本实施例中,预设的处理模型可以包括美化处理算法,例如,在实时磨皮的应用场景中,可以采用双边滤波的美化处理算法,实现实时的磨皮图像效果。
需要说明的是,处理预设的处理模型对应的至少一个区域图像,包括处理一个、两个或多个区域图像。进一步地,各区域图像的处理模型可以相同也可以不同。具体地,可以根据用户输入的选中指令,获取至少一个区域图像;基于预设的处理模型对选中的至少一个区域图像进行处理。
S206、对处理后的区域图像进行高斯平滑处理,获取所述区域图像各像素点的平滑像素值。
具体地,在RGB图像分割时,可以使得分割后的RGB图像的大小与原始RGB图像的大小一致,分割后的RGB图像的像素点与原始RGB图像的像素点一一对应,进一步地,分割后的RGB图像中的每个像素点的像素值可仅采用1或者0
表示,分别代表该像素所对应的RGB图像的位置是否为要处理的区域图像。如上,如果分割成2个区域图像,可以采用1表示前景区域;0表示背景区域。在此基础上,对上述分割图像进行高斯平滑,这样得到的分割图像的像素值a(x,y)则可能取0-1之间的任意值。该操作可以有效去除图像中的噪声,保证图像品质。
S207、根据所述平滑像素值与所述RGB图像各像素点的像素值,计算融合图像的融合像素值。
示例性地,可以将平滑像素值与RGB图像各像素点的像素值进行加权后求和,计算出融合图像的融合像素值。举例而言,假设处理后的区域图像的平滑像素值为IB(x,y),原始RGB图像各像素点的像素值为I0(x,y),此时,可以将IB(x,y)与I0(x,y)进行加权后,求和,得到最终的融合图像的融合像素值IR(x,y),即IR(x,y)=IB(x,y)*a(x,y)+I0(x,y)*{1-a(x,y)}。其中,a(x,y)表示平滑像素值的权重。可以理解的是,a(x,y)的具体取值可以根据实际情况设置,可以为固定值,也可以通过其他参数计算得出,在此并不做限定。
S208、根据所述融合像素值输出融合图像。
本实施例的技术方案,通过同一场景下的深度图像的各像素点的第一深度值,确定RGB图像各像素点的第二深度值,能够通过深度图像与RGB图像获取更多的场景信息,通过对深度图像的分割完成对RGB图像的分割,能够提高区域图像分割的精度;进而根据第二深度值和预设的规则分割RGB图像,处理预设的处理模型对应的至少一个区域图像,能够将图像根据自身需求将图像分割为至少一个区域图像,且能够在至少一个区域图像中选择要处理的一个或多个区域图像,而后基于预设的处理模型处理对应的区域图像,不仅能够使得用户
根据自身的需求实现对图像的局部处理,且可以满足用户对不同区域图像进行不同处理的个性化需求;最后将处理后的区域图像的像素值与原始的RGB图像的像素值融合生成最终的融合图像的融合像素值,输出图像,同时兼顾计算效率与融合效果。
实施例三
图3为本实施例三所提供的一种图像处理装置的结构示意图。该装置可通过硬件和/或软件的方式实现,并一般可独立的配置在终端中实现本实施例的方法。如图3所示,所述图像处理装置具体包括:原始图像获取模块310、区域图像处理模块320和图像融合模块330。
其中,原始图像获取模块310,用于获取同一场景下的深度图像和RGB图像;区域图像处理模块320,用于根据所述深度图像中的深度值分割所述RGB图像得到至少两个区域图像,处理至少一个所述区域图像;图像融合模块330,用于将处理后的区域图像与所述RGB图像进行融合。
在上述各实施例的基础上,区域图像处理模块320可以包括:深度值确定子模块、区域图像生成子模块和区域图像处理子模块。
其中,深度值确定子模块,用于根据所述深度图像的各像素点的第一深度值,确定所述RGB图像各像素点对应的第二深度值;区域图像生成子模块,用于根据所述第二深度值和预设的规则分割所述RGB图像得到至少两个区域图像;区域图像处理子模块,用于处理预设的处理模型对应的至少一个所述区域图像。
在上述各实施例的基础上,所述深度值确定子模块还可以包括映射矩阵计算单元和第二深度值计算单元。
其中,映射矩阵计算单元,用于计算所述深度图像的各像素点的第一坐标与所述RGB图像各像素点的第二坐标的映射矩阵;第二深度值计算单元,用于根据所述第二坐标、所述映射矩阵以及所述深度图像各像素点的第一深度值,确定所述RGB图像各像素点对应的第二深度值。
在上述各实施例的基础上,所述区域图像生成子模块具体用于:判断所述第二深度值是否大于预设的深度阈值,若是,则将与所述第二深度值对应的像素点的位置作为前景区域,若否,则将与所述第二深度值对应的像素点的位置作为背景区域;根据所述前景区域和所述背景区域,分割所述RGB图像得到两个区域图像。
在上述各实施例的基础上,所述区域图像的各像素点的像素值分别用1或者0表示;其中,1表示所述前景区域;0表示所述背景区域。
在上述各实施例的基础上,图像融合模块330具体用于:对处理后的所述区域图像进行高斯平滑处理,获取所述区域图像各像素点的平滑像素值;根据所述平滑像素值与所述RGB图像各像素点的像素值,计算融合图像的融合像素值;根据所述融合像素值输出融合图像。
在上述各个实施例中,所述预设的处理模型可以包括美化处理算法,例如,在实时磨皮的应用场景中,可以采用双边滤波的美化处理算法,实现实时的磨皮图像效果。
本实施例还提供了一种终端,所述终端包括本发明任意实施例所述的图像处理装置。示例性地,所述终端可以包括手机、平板电脑、智能手表及照相机等具有拍摄功能的设备。
上述实施例中提供的图像处理装置及移动终端可执行本发明任意实施例所
提供的图像处理方法,具备执行该方法相应的功能模块和有益效果。未在上述实施例中详尽描述的技术细节,可参见本发明任意实施例所提供的图像处理方法。
注意,上述仅为本发明的较佳实施例及所运用技术原理。本领域技术人员会理解,本发明不限于这里所述的特定实施例,对本领域技术人员来说能够进行各种明显的变化、重新调整和替代而不会脱离本发明的保护范围。因此,虽然通过以上实施例对本发明进行了较为详细的说明,但是本发明不仅仅限于以上实施例,在不脱离本发明构思的情况下,还可以包括更多其他等效实施例,而本发明的范围由所附的权利要求范围决定。
Claims (10)
- 一种图像处理方法,其特征在于,包括:获取同一场景下的深度图像和RGB图像;根据所述深度图像中的深度值分割所述RGB图像得到至少两个区域图像,处理至少一个所述区域图像;将处理后的区域图像与所述RGB图像进行融合。
- 根据权利要求1所述的图像处理方法,其特征在于,所述根据所述深度图像中的深度值分割所述RGB图像得到至少两个区域图像,处理至少一个所述区域图像,包括:根据所述深度图像的各像素点的第一深度值,确定所述RGB图像各像素点对应的第二深度值;根据所述第二深度值和预设的规则分割所述RGB图像得到至少两个区域图像;处理预设的处理模型对应的至少一个所述区域图像。
- 根据权利要求2所述的图像处理方法,其特征在于,所述根据所述深度图像的各像素点的第一深度值,确定所述RGB图像各像素点对应的第二深度值包括:计算所述深度图像的各像素点的第一坐标与所述RGB图像各像素点的第二坐标的映射矩阵;根据所述第二坐标、所述映射矩阵以及所述深度图像各像素点的第一深度值,确定所述RGB图像各像素点对应的第二深度值。
- 根据权利要求2或3所述的图像处理方法,其特征在于,所述根据所述第二深度值和预设的规则分割所述RGB图像得到至少两个区域图像,包括:判断所述第二深度值是否大于预设的深度阈值,若是,则将与所述第二深度值对应的像素点的位置作为前景区域,若否,则将与所述第二深度值对应的像素点的位置作为背景区域;根据所述前景区域和所述背景区域,分割所述RGB图像得到两个区域图像。
- 根据权利要求4所述的图像处理方法,其特征在于,所述区域图像的各像素点的像素值分别用1或者0表示;其中,1表示所述前景区域;0表示所述背景区域。
- 根据权利要求5所述的图像处理方法,其特征在于,所述将处理后的区域图像与所述RGB图像进行融合包括:对处理后的所述区域图像进行高斯平滑处理,获取所述区域图像各像素点的平滑像素值;根据所述平滑像素值与所述RGB图像各像素点的像素值,计算融合图像的融合像素值;根据所述融合像素值输出融合图像。
- 一种图像处理装置,其特征在于,包括:原始图像获取模块,用于获取同一场景下的深度图像和RGB图像;区域图像处理模块,用于根据所述深度图像中的深度值分割所述RGB图像得到至少两个区域图像,处理至少一个所述区域图像;图像融合模块,用于将处理后的区域图像与所述RGB图像进行融合。
- 根据权利要求7所述的图像处理装置,其特征在于,所述区域图像处理模块包括:深度值确定子模块,用于根据所述深度图像的各像素点的第一深度值,确 定所述RGB图像各像素点对应的第二深度值;区域图像生成子模块,用于根据所述第二深度值和预设的规则分割所述RGB图像得到至少两个区域图像;区域图像处理子模块,用于处理预设的处理模型对应的至少一个所述区域图像。
- 根据权利要求8所述的图像处理装置,其特征在于,所述深度值确定子模块包括:映射矩阵计算单元,用于计算所述深度图像的各像素点的第一坐标与所述RGB图像各像素点的第二坐标的映射矩阵;第二深度值计算单元,用于根据所述第二坐标、所述映射矩阵以及所述深度图像各像素点的第一深度值,确定所述RGB图像各像素点对应的第二深度值。
- 根据权利要求8所述的图像处理装置,其特征在于,所述区域图像生成子模块具体用于:判断所述第二深度值是否大于预设的深度阈值,若是,则将与所述第二深度值对应的像素点的位置作为前景区域,若否,则将与所述第二深度值对应的像素点的位置作为背景区域;根据所述前景区域和所述背景区域,分割所述RGB图像得到两个区域图像。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610890902.6A CN106447677A (zh) | 2016-10-12 | 2016-10-12 | 图像处理方法和装置 |
| CN201610890902.6 | 2016-10-12 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2018068420A1 true WO2018068420A1 (zh) | 2018-04-19 |
Family
ID=58173811
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2016/113270 Ceased WO2018068420A1 (zh) | 2016-10-12 | 2016-12-29 | 图像处理方法和装置 |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN106447677A (zh) |
| WO (1) | WO2018068420A1 (zh) |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106991370B (zh) * | 2017-02-28 | 2020-07-31 | 中科唯实科技(北京)有限公司 | 基于颜色和深度的行人检索方法 |
| CN107316281B (zh) * | 2017-06-16 | 2021-03-02 | Oppo广东移动通信有限公司 | 图像处理方法、装置和终端设备 |
| CN107369165A (zh) * | 2017-07-10 | 2017-11-21 | Tcl移动通信科技(宁波)有限公司 | 一种视频选择画面优化方法及存储介质、智能终端 |
| CN107301408B (zh) * | 2017-07-17 | 2020-06-23 | 成都通甲优博科技有限责任公司 | 人体掩膜提取方法及装置 |
| CN107368810A (zh) * | 2017-07-20 | 2017-11-21 | 北京小米移动软件有限公司 | 人脸检测方法及装置 |
| CN107507239B (zh) * | 2017-08-23 | 2019-08-20 | 维沃移动通信有限公司 | 一种图像分割方法及移动终端 |
| CN109867077A (zh) * | 2017-12-04 | 2019-06-11 | 北京京东尚科信息技术有限公司 | 用于仓储拣货的系统、方法、装置、拣选车及穿梭车 |
| CN107945192B (zh) * | 2017-12-14 | 2021-10-22 | 北京信息科技大学 | 一种托盘纸箱垛型实时检测方法 |
| CN109447912B (zh) * | 2018-10-18 | 2022-04-19 | 广东欧谱曼迪科技有限公司 | 一种荧光导航内窥镜系统的荧光图像自适应增强及降噪方法 |
| CN109636814A (zh) * | 2018-12-18 | 2019-04-16 | 联想(北京)有限公司 | 一种图像处理方法及电子设备 |
| CN110378852A (zh) * | 2019-07-11 | 2019-10-25 | 北京奇艺世纪科技有限公司 | 图像增强方法、装置、计算机设备和存储介质 |
| CN112313944A (zh) * | 2019-11-28 | 2021-02-02 | 深圳市大疆创新科技有限公司 | 图像处理方法、装置、设备及存储介质 |
| CN111458693A (zh) * | 2020-02-01 | 2020-07-28 | 上海鲲游光电科技有限公司 | 直接测距tof分区探测方法及其系统和电子设备 |
| CN111683192B (zh) * | 2020-06-11 | 2023-03-21 | 展讯通信(上海)有限公司 | 图像处理方法及相关产品 |
| CN111866476B (zh) * | 2020-08-31 | 2023-04-07 | 维沃移动通信有限公司 | 图像拍摄方法、装置及电子设备 |
| CN113379854B (zh) * | 2021-05-31 | 2022-12-06 | 上海集成电路制造创新中心有限公司 | 相机图像融合方法及相机图像融合系统 |
| CN115760668A (zh) * | 2022-11-30 | 2023-03-07 | 昆山丘钛微电子科技股份有限公司 | 一种相机图像融合方法、装置、介质及设备 |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102379127A (zh) * | 2009-03-31 | 2012-03-14 | 夏普株式会社 | 影像处理装置、影像处理方法以及计算机程序 |
| CN102999901A (zh) * | 2012-10-17 | 2013-03-27 | 中国科学院计算技术研究所 | 基于深度传感器的在线视频分割后的处理方法及系统 |
| US20150002545A1 (en) * | 2013-06-28 | 2015-01-01 | Canon Kabushiki Kaisha | Variable blend width compositing |
| CN105430295A (zh) * | 2015-10-30 | 2016-03-23 | 努比亚技术有限公司 | 图像处理装置及方法 |
| CN105791796A (zh) * | 2014-12-25 | 2016-07-20 | 联想(北京)有限公司 | 图像处理方法和图像处理装置 |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103795961A (zh) * | 2012-10-30 | 2014-05-14 | 三亚中兴软件有限责任公司 | 会议电视网真系统及其图像处理方法 |
| CN105869159A (zh) * | 2016-03-28 | 2016-08-17 | 联想(北京)有限公司 | 一种图像分割方法及装置 |
-
2016
- 2016-10-12 CN CN201610890902.6A patent/CN106447677A/zh active Pending
- 2016-12-29 WO PCT/CN2016/113270 patent/WO2018068420A1/zh not_active Ceased
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102379127A (zh) * | 2009-03-31 | 2012-03-14 | 夏普株式会社 | 影像处理装置、影像处理方法以及计算机程序 |
| CN102999901A (zh) * | 2012-10-17 | 2013-03-27 | 中国科学院计算技术研究所 | 基于深度传感器的在线视频分割后的处理方法及系统 |
| US20150002545A1 (en) * | 2013-06-28 | 2015-01-01 | Canon Kabushiki Kaisha | Variable blend width compositing |
| CN105791796A (zh) * | 2014-12-25 | 2016-07-20 | 联想(北京)有限公司 | 图像处理方法和图像处理装置 |
| CN105430295A (zh) * | 2015-10-30 | 2016-03-23 | 努比亚技术有限公司 | 图像处理装置及方法 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106447677A (zh) | 2017-02-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2018068420A1 (zh) | 图像处理方法和装置 | |
| US9378583B2 (en) | Apparatus and method for bidirectionally inpainting occlusion area based on predicted volume | |
| US9898856B2 (en) | Systems and methods for depth-assisted perspective distortion correction | |
| CN114372931B (zh) | 一种目标对象虚化方法、装置、存储介质及电子设备 | |
| WO2018082185A1 (zh) | 图像处理方法和装置 | |
| CN104205826B (zh) | 用于重建高密度三维图像的设备和方法 | |
| CN106570472B (zh) | 一种肤色检测方法、装置及终端 | |
| Luo et al. | A disocclusion inpainting framework for depth-based view synthesis | |
| US10154241B2 (en) | Depth map based perspective correction in digital photos | |
| CN105741231A (zh) | 图像的美肤处理方法和装置 | |
| JP2010140097A (ja) | 画像生成方法、画像認証方法、画像生成装置、画像認証装置、プログラム、および記録媒体 | |
| CN104809687A (zh) | 三维人脸图像生成方法及系统 | |
| CN111476812A (zh) | 地图分割方法、装置、位姿估计方法和设备终端 | |
| US20220101532A1 (en) | Method and device for performing plane detection | |
| CN114119701A (zh) | 图像处理方法及其装置 | |
| US9171357B2 (en) | Method, apparatus and computer-readable recording medium for refocusing photographed image | |
| US12374117B2 (en) | Method, system and computer readable media for object detection coverage estimation | |
| CN107564085B (zh) | 图像扭曲处理方法、装置、计算设备及计算机存储介质 | |
| CN110188640B (zh) | 人脸识别方法、装置、服务器和计算机可读介质 | |
| US12051226B2 (en) | Feature detection methods and systems using deconstructed color image data | |
| CN112053434B (zh) | 视差图的生成方法、三维重建方法及相关装置 | |
| KR102267442B1 (ko) | 가변 시점에서 촬영된 카메라 왜곡 보정 방법 및 이를 이용한 블록 3차원 모델링 방법 | |
| Khan et al. | Offset aperture: A passive single-lens camera for depth sensing | |
| CN115049711A (zh) | 图像配准方法、装置、电子设备及介质 | |
| WO2020118565A1 (en) | Keyframe selection for texture mapping wien generating 3d model |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16918604 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 16918604 Country of ref document: EP Kind code of ref document: A1 |