WO2018149350A1 - 一种非面部roi识别方法及装置 - Google Patents

一种非面部roi识别方法及装置 Download PDF

Info

Publication number
WO2018149350A1
WO2018149350A1 PCT/CN2018/075675 CN2018075675W WO2018149350A1 WO 2018149350 A1 WO2018149350 A1 WO 2018149350A1 CN 2018075675 W CN2018075675 W CN 2018075675W WO 2018149350 A1 WO2018149350 A1 WO 2018149350A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
face
facial
processed
roi
Prior art date
Application number
PCT/CN2018/075675
Other languages
English (en)
French (fr)
Inventor
雷宇
金宇林
伏英娜
Original Assignee
迈吉客科技（北京）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 迈吉客科技（北京）有限公司 filed Critical 迈吉客科技（北京）有限公司
Publication of WO2018149350A1 publication Critical patent/WO2018149350A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/88—Image or video recognition using optical means, e.g. reference filters, holographic masks, frequency domain filters or spatial domain filters

Definitions

  • the present invention relates to the field of image processing technologies, and in particular, to a method and apparatus for non-face ROI recognition.
  • facial expression recognition technology is applied to various fields, such as animation production, film production, video chat, etc., by identifying the user's facial expressions, creating user facial expressions and animations.
  • identifying non-face ROI such as tongue, teeth, and eyeball is the focus and difficulty of facial expression recognition.
  • the identification of non-facial ROI is mainly performed by a method of SVM (Support Vector Machine) and a neural network algorithm.
  • SVM Small Vector Machine
  • SVM For the current method of identifying non-face ROIs through SVM and neural network algorithms, SVM needs to raise the histogram data to a higher dimension for calculation, and the neural network algorithm needs to perform multiple convolution volumes and pools on the image. Processing. Therefore, both SVM and neuron network algorithms take a long time to process large amounts of data, resulting in a slower rate of recognition of non-face ROIs.
  • Embodiments of the present invention provide a method and apparatus for non-face ROI identification, which can improve the rate of recognition of a non-face ROI.
  • an embodiment of the present invention provides a method for non-face ROI identification, including:
  • the extracting the facial image from the image to be processed by the at least one color sample region comprises:
  • An image located in the combined face region is extracted from the image to be processed as the face image.
  • the mask image is mask-removed to obtain a target image corresponding to the non-face ROI, including:
  • the identifying the non-face ROI in the image to be processed according to the at least one facial key point and the target image comprises:
  • Whether the remaining image is the non-face ROI is identified according to a position, a shape, and a size of a remaining image on the target image with respect to each of the face key points.
  • the non-face ROI comprises any one of a tongue, a tooth, an eyeball and a beard.
  • the embodiment of the present invention further provides a non-face ROI identification device, including: an extracting unit, a eliminating unit, and an identifying unit;
  • the extracting unit is configured to determine at least one facial key point from the image to be processed, determine at least one color sample region according to the at least one facial key point, and the image to be processed by the at least one color sample region Extracting a facial image;
  • the eliminating unit is configured to perform mask elimination on the facial image extracted by the extracting unit, and obtain a target image corresponding to the non-face ROI on the facial image;
  • the identifying unit is configured to identify the non-face ROI in the image to be processed according to the at least one facial key point determined by the extracting unit and the target image acquired by the eliminating unit.
  • the extracting unit includes: acquiring a subunit
  • the acquiring sub-unit performs color statistics on each of the at least one color sample region, obtains a sample color corresponding to the color sample region, and determines that the sample color is in an adjacent region of the color sample region Normalizing the frequency of occurrence, and determining the adjacent region whose normalized appearance frequency is greater than a preset threshold as the face region corresponding to the color sample region; and each of the color sample regions and the corresponding facial region Combining to form a combined face region; and extracting an image located in the combined face region from the image to be processed as the face image.
  • the eliminating unit is configured to perform mask elimination on the facial image, and obtain a target image corresponding to the non-face ROI from an image to be processed that cancels the facial image according to the at least one facial key point .
  • the identifying unit is configured to determine, according to the at least one facial key point, an area where the interference image is located on the target image, and eliminate the interference image; and according to each of the remaining images on the target image The position, shape and size of the face key point, identifying whether the remaining image is the non-face ROI.
  • the non-face ROI includes any one of a tongue, a tooth, an eyeball, and a beard.
  • the embodiment of the present invention further provides a readable medium, including executing instructions, when the processor of the storage controller executes the execution instruction, the storage controller executes the non-face ROI provided by the foregoing embodiment.
  • the method of identification when the processor of the storage controller executes the execution instruction, the storage controller executes the non-face ROI provided by the foregoing embodiment.
  • an embodiment of the present invention further provides a storage controller, including: a processor, a memory, and a bus;
  • the memory is configured to store an execution instruction
  • the processor is connected to the memory through the bus, and when the storage controller is running, the processor executes the execution instruction stored in the memory to make
  • the storage controller performs the method of non-face ROI identification provided by the above embodiments.
  • Embodiments of the present invention provide a non-face ROI identification method, apparatus, and readable medium and storage controller. After determining at least one facial key point in an image to be processed, determining at least one color according to the determined facial key point In the sample area, the face image is extracted from the image to be processed through each color sample area, and the target image that may be generated by the non-face ROI is obtained by masking the face image, and then according to each face key point and the target image to be processed. The non-face ROI is identified. It can be seen that when the non-face ROI is recognized, only the target image of the non-face ROI may be obtained after the mask image is removed, and the non-face ROI is included in the target image according to each facial key point. The amount of data to be processed is small, so that the rate of recognition of non-face ROIs can be increased.
  • FIG. 1 is a flowchart of a method for non-face ROI recognition according to an embodiment of the present invention
  • FIG. 2 is a flowchart of a method for non-face ROI identification according to another embodiment of the present invention.
  • FIG. 3 is a schematic diagram of an apparatus for a device for non-face ROI identification according to an embodiment of the present invention.
  • FIG. 4 is a schematic diagram of a device for non-face ROI recognition according to an embodiment of the present invention.
  • an embodiment of the present invention provides a method for non-face ROI identification, which may include the following steps:
  • Step 101 Determine at least one facial key point from the image to be processed
  • Step 102 Determine at least one color sample region according to the at least one facial key point
  • Step 103 Extract a facial image from the image to be processed by using the at least one color sample region;
  • Step 104 performing mask elimination on the facial image to obtain a target image corresponding to the non-face region of interest ROI;
  • Step 105 Identify the non-face ROI in the image to be processed according to the at least one facial key point and the target image.
  • An embodiment of the present invention provides a method for non-face ROI identification. After determining at least one facial key point in an image to be processed, determining at least one color sample region according to the determined facial key point, and waiting for each color sample region The facial image is extracted on the processed image, and the target image that may be generated by the non-face ROI is obtained by masking the facial image, and then the non-face ROI in the image to be processed is identified according to each facial key point and the target image. It can be seen that when the non-face ROI is recognized, only the target image of the non-face ROI may be obtained after the mask image is removed, and the non-face ROI is included in the target image according to each facial key point. The amount of data to be processed is small, so that the rate of recognition of non-face ROIs can be increased.
  • the face image when the face image is extracted from the image to be processed by the color sample region in step 103, color statistics are performed on the color sample region for each color sample region, and the color sample region is obtained.
  • a sample color determining a normalized appearance frequency of the sample color in an adjacent region adjacent to the color sample region, and determining an adjacent region whose normalized appearance frequency is greater than a preset value as the color sample region corresponding to The facial area.
  • a combined face region is formed, and an image located in the combined face region is extracted from the image to be processed as a face image.
  • two color sample regions are determined on the image to be processed, and the two color sample regions are respectively the regions of the two tibia on the face image, respectively, the color sample region 1 and the color.
  • Sample area 2 Color statistic is performed on the color sample area 1, and the color sample 1 in the color sample area 1 is obtained, and the normalized appearance frequency of the color sample 1 in the area adjacent to the color sample area 1 is determined, and the normalized appearance frequency is greater than the preset.
  • the adjacent area of the threshold of 80% is determined as the face area corresponding to the color sample area 1. Accordingly, the face region corresponding to the color sample region 2 is determined by the same method. After combining the face images corresponding to the color sample area 1 and the color sample area 2, a combined face area is formed, and an image located in the combined face area on the image to be processed is taken as a face image.
  • At least one color sample area may be determined on the facial image according to each facial key point, and color sample analysis of the color sample area is performed, and the sample color corresponding to each color sample area is counted. . Determining a normalized appearance frequency of the color sample corresponding to the color sample region in the image to be processed for each color sample region, and adjacent to the color sample region on the image to be processed and normalizing the occurrence frequency to be greater than a preset threshold The area is determined as the face area corresponding to the color sample area. After the face regions corresponding to the respective color sample regions are combined, the combined face region formed covers the entire face, and the image to be processed located in the combined face region is the face image.
  • the non-face ROI Since there is a large difference between the color of the non-face ROI and the face image, at least one color sample area is determined on the face image, and an area close to the color of the color sample area is obtained as a face area by determining a normalized appearance frequency, and each color is used.
  • the face regions corresponding to the sample regions are combined, and the image to be processed in the combined face region is a face image, and the face image is extracted.
  • the non-face ROI on the image to be processed can be conveniently determined after determining the face image.
  • the face regions corresponding to different color sample regions may have overlapping portions.
  • positioning is performed by each facial key point, and the face regions corresponding to the same region are overlapped. deal with.
  • the face image on the image to be processed is subjected to eye drenching cancellation, and the image to be processed of the face image is eliminated.
  • a target image corresponding to the non-face ROI is obtained according to each facial key point.
  • the non-face ROI is a tongue
  • the face image on the image to be processed is mask-removed.
  • the area where the tongue may appear is positioned according to the previously determined at least one face key point, and the target image including the lips and possibly including the beard and the tongue is acquired.
  • the non-face ROI when the non-face ROI is identified according to the face key point and the target image in step 105, the area where the interference image is located on the target image is located according to the previously determined at least one face key point. And the interference image is eliminated. For the remaining image remaining after the interference image is removed on the target image, according to the position of the remaining image relative to each facial key point, and the shape of the remaining image and the size of the facial image, it is determined that the image is removed after the interference image is eliminated. Whether the remaining image is the non-face ROI to be identified.
  • the non-face ROI is a tongue
  • the acquired target image includes an interference image such as a lip
  • the at least one facial key point determined before may be used to locate an interference image such as a lip, and determine an interference image such as a lip on the target image. After that, the interference image will be eliminated.
  • the target image further includes the remaining image after eliminating the interference image such as the lips, further determining whether the remaining image is a tongue according to the position, shape, and size of the remaining image, and if so, determining that the image to be processed includes the tongue, that is, including the non-face ROI, otherwise It is determined that the non-face ROI is not included on the image to be processed; if the target image is completely eliminated after the interference image such as the lips is eliminated, it is determined that the non-face ROI is not included on the image to be processed.
  • the non-face ROI may be any of a tongue, a tooth, an eyeball, and a beard.
  • the non-face ROI recognition method provided by the embodiment of the present invention can identify a plurality of non-face ROI regions, thereby improving the applicability of the non-face ROI recognition method.
  • the method for non-face ROI identification provided by the embodiment of the present invention is further described in detail by taking the tongue as a non-face ROI as an example. As shown in FIG. 2, the method may include the following steps:
  • Step 201 Acquire an image to be processed.
  • the image to be processed is acquired in real time.
  • each frame image in the video stream is treated as a to-be-processed image.
  • the video data of the user A is collected by the camera, and each frame image included in the collected video data is used as a to-be-processed image to acquire a first frame to be processed image.
  • Step 202 Determine whether a face image is included in the image to be processed, and if yes, execute step 203, otherwise perform step 201.
  • the image to be processed is subjected to face recognition to identify whether the image to be processed includes a face image, and if yes, step 203 is performed to further identify whether the image to be processed includes non- Face ROI; if no, step 201 is performed to acquire the next image to be processed.
  • step 201 is performed to acquire the next frame to be processed image collected by the camera, and the second frame to be processed image.
  • Step 203 Determine at least one facial key point on the image to be processed.
  • At least one facial key point is determined on the image to be processed, wherein the facial key point is located on the facial image included in the image to be processed.
  • the image A to be processed includes a face image
  • three face key points are determined on the image to be processed A, wherein the face key point a is the position of the left eye on the face image of the user A, and the face key point b is the user A.
  • the position of the right eye on the face image, the key point c of the face is the position of the nose tip on the face image of the user A.
  • Step 204 Determine at least one color sample region according to each facial key point.
  • At least one color sample region is determined on the facial image included in the image to be processed according to the at least one facial key determined in step 203.
  • two color sample regions are determined on the face image of the user A included in the image A to be processed, wherein the color sample region d is the region where the left tibia on the face image of the user A is located, and the color sample region e is the user A The area of the facial image on the right side of the humerus.
  • Step 205 Extract a facial image from the image to be processed according to the at least one color sample region.
  • a sample color corresponding to the color sample region is obtained by color analysis, and a normalized appearance frequency of the sample color in each region of the image to be processed is determined.
  • the image to be processed is adjacent to the color sample region, and the region where the frequency of occurrence is greater than a preset threshold is used as the face region corresponding to the color sample region.
  • the face regions corresponding to the respective color sample regions are combined to form a combined face region, and an image located in the combined face region on the image to be processed is determined as a face image.
  • the color X is determined as the sample color of the color sample region d; and the color X is determined in each region of the entire image to be processed A.
  • the normalized appearance frequency, the area of the image to be processed A adjacent to the color sample area d and the normalized appearance frequency of the color X being greater than the preset threshold value of 80% is determined as the face area corresponding to the color sample area d, such as determining
  • the face area corresponding to the color sample area d is the area of the left half of the user A on the image A to be processed, the forehead, the chin, and the nose.
  • the face area corresponding to the color sample area e is determined by the same method as the area of the right half face and the forehead of the user A on the image A to be processed.
  • the combined face region includes other face regions on the image A to be processed other than the eyes and mouth of the user A.
  • An image located within the combined face region is extracted from the image to be processed A as a face image, that is, a face image of the user A other than the eyes and the mouth is extracted.
  • Step 206 Perform mask elimination on the face image to obtain a target area that may include a non-face ROI.
  • the facial image on the image to be processed is mask-removed, and the target image on the image to be processed that may include the non-face ROI is obtained after the facial image is eliminated.
  • a mouth image which may include the tongue image of the user A is obtained, and the mouth image is determined as the target image.
  • Step 207 Eliminate the interference image in the target image according to each facial key point, and obtain the remaining image.
  • the interference image existing in the target image is located, and after the interference image in the target image is eliminated, the possible non-face may be obtained.
  • the remaining image of the ROI is obtained.
  • the interference image (lip image) on the target image is positioned by the face key point a (left eye position), the face key point b (eye position), and the face key point c (nose tip position), After the area of the lip image is determined, the lip image on the target image is eliminated, and the target image after the lip image is eliminated is taken as the remaining image.
  • Step 208 Determine whether the remaining image is a non-face ROI according to the position, shape and size of the remaining image.
  • whether the remaining images are non-face ROIs is determined according to the relative positions of the remaining images and the respective face key points, and the shape of the remaining images and the size of the face image.
  • an image is further included on the target image after the interference image is eliminated, that is, there is a remaining image
  • the position of the remaining image with respect to the face key point a, the face key point b, and the face key point c is determined, and it is determined whether the remaining image is located at the tongue.
  • a possible position if yes, further determining whether the shape and size of the remaining image match the image of the tongue, and if so, determining that the image of the tongue exists on the image to be processed A, and displaying the tongue on the display device of the user B side Corresponding effects.
  • the target image after the interference image is eliminated is completely eliminated, or although the remaining image exists, but the position, shape or size of the remaining image does not win with the position, shape and size of the tongue, it is determined that the tongue is present in the upper portion of the image to be processed A.
  • an embodiment of the present invention provides a device for non-face ROI identification.
  • the device embodiment may be implemented by software, or may be implemented by hardware or a combination of hardware and software. From a hardware level, as shown in FIG. 3, a hardware structure diagram of a device for a device for non-face ROI identification provided by an embodiment of the present invention, except for the processor, the memory, the network interface, and the non-easy device shown in FIG.
  • the device in which the device is located in the embodiment may also include other hardware, such as a forwarding chip responsible for processing the message, and the like. Taking the software implementation as an example, as shown in FIG.
  • the apparatus for non-face ROI identification includes: an extracting unit 401, a eliminating unit 402, and an identifying unit 403;
  • the extracting unit 401 is configured to determine at least one facial key point from the image to be processed, and determine at least one color sample region according to the at least one facial key point, by using the at least one color sample region from the to-be-processed Extracting a facial image from the image;
  • the eliminating unit 402 is configured to perform mask elimination on the facial image extracted by the extracting unit 401, and obtain a target image corresponding to the non-face ROI on the facial image;
  • the identifying unit 403 is configured to: according to the at least one facial key point determined by the extracting unit 401 and the target image acquired by the eliminating unit 402, the non-face ROI in the image to be processed Identify.
  • the extracting unit 401 includes an obtaining subunit
  • the obtaining subunit is configured to perform color statistics on the color sample region for each of the color sample regions, obtain a sample color corresponding to the color sample region, and determine that the sample color is in an area adjacent to the color sample region. Normalizing the frequency of occurrence, and determining the adjacent region whose normalized appearance frequency is greater than a preset threshold as the face region corresponding to the color sample region; and performing each of the color sample regions and the corresponding facial region Combining to form a combined face region; and extracting an image located in the combined face region from the image to be processed as the face image.
  • the eliminating unit 402 is configured to perform mask elimination on the facial image, and obtain the image from the image to be processed that eliminates the facial image according to the at least one facial key point.
  • the identifying unit 403 is configured to determine, according to the at least one facial key point, an area where the interference image is located on the target image, and eliminate the interference image; and according to the target image The remaining image is identified with respect to the position, shape and size of each of the face key points, and whether the remaining image is the non-face ROI.
  • the embodiment of the present invention further provides a readable medium, including executing instructions, when the processor of the storage controller executes the execution instruction, the storage controller performs the non-face ROI identification method provided by the foregoing embodiment.
  • An embodiment of the present invention further provides a storage controller, including: a processor, a memory, and a bus;
  • the memory is configured to store an execution instruction
  • the processor is connected to the memory through the bus, and when the storage controller is running, the processor executes the execution instruction stored in the memory to make
  • the storage controller performs the method of non-face ROI identification provided by the above embodiments.
  • the method and apparatus for non-face ROI identification provided by the embodiment of the present invention, after determining at least one facial key point in the image to be processed, determining at least one color sample area according to the determined facial key point, and passing each color
  • the sample area extracts the face image from the image to be processed, and the target image of the non-face ROI may be obtained by masking the face image, and then the non-face ROI in the image to be processed is identified according to each face key point and the target image.
  • the non-face ROI is recognized, only the target image of the non-face ROI may be obtained after the mask image is removed, and the non-face ROI is included in the target image according to each facial key point.
  • the amount of data to be processed is small, so that the rate of recognition of non-face ROIs can be increased.
  • the face image is eliminated by color analysis, thereby identifying whether the remaining image is a non-face ROI, and identifying the non-face ROI.
  • the process depends on the color of the image, so that the non-face ROI can be recognized in various environments, such as over-exposure of the image to be processed, underexposure, etc., which improves the applicability of the non-face ROI recognition method and apparatus.
  • the facial image is eliminated by color analysis and statistical methods, thereby identifying whether there is a non-face ROI in the image to be processed, compared with the existing SVM and
  • the neural network algorithm improves the rate of identifying non-face ROIs, and ensures the timeliness of identifying non-face ROIs, so that the non-face ROI recognition method and device can be applied to occasions with high real-time requirements, further improving the non- Applicability of facial ROI recognition methods and devices.
  • the non-face ROI may be any one of a tongue, a tooth, an eyeball and a beard, so the method and device for identifying the non-face ROI can satisfy different The user's needs improve the versatility of the non-face ROI recognition method and apparatus.
  • the disclosed systems, devices, and methods may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the non-face ROI recognition method and apparatus avoids processing of a large number of redundant image signals, and simplifies the calculation amount of image processing. It is only necessary to perform mask elimination on the face image to obtain a target image in which a non-face ROI may occur, and further, according to each face key point, whether or not the non-face ROI is included in the target image, the amount of data to be processed is small, thereby improving the right The rate at which the facial ROI is recognized.
  • the non-face ROI identification method and apparatus of the embodiments of the present invention determine that the non-face ROI can be generally applied to the smart mobile terminal device to improve the efficiency of the human-computer interaction process.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本发明提供了一种非面部ROI识别的方法及装置，该方法包括：从待处理的图像中确定至少一个面部关键点；根据所述至少一个面部关键点，确定至少一个颜色样本区域；通过所述至少一个颜色样本区域，从所述待处理的图像中提取面部图像；对所述面部图像进行掩膜消除，获得非面部感兴趣区域ROI对应的目标图像；根据所述至少一个面部关键点及所述目标图像，对所述待处理的图像中的所述非面部ROI进行识别。该装置包括：提取单元、消除单元及识别单元。本方案能够提高对非面部ROI进行识别的速率。

Description

一种非面部ROI识别方法及装置
本发明是要求由申请人提出的，申请日为2017年02月20日，申请号为CN201710090091.6，名称为“一种非面部ROI识别方法及装置”的申请的优先权。以上申请的全部内容通过整体引用结合于此。
技术领域
本发明涉及图像处理技术领域，特别涉及一种非面部ROI识别的方法及装置。
背景技术
随着计算机技术和图像处理技术的不断发展与进步，面部表情识别技术被应用于各个领域，比如动画制作、影视制作、视频聊天等，通过对用户的面部表情进行识别，创建用户面部表情与动画人物面部表情的映射关系，以进行动画片、科幻电影的制作，趣味视频聊天或趣味照片拍摄。在面部表情识别过程中，识别舌头、牙齿、眼球等非面部ROI(Region Of Interest，感兴趣区域)是面部表情识别的重点和难点。
目前，识别非面部ROI主要通过SVM(Support Vector Machine，支持向量机分类器)和神经元网络算法的方法进行。
针对于目前通过SVM和神经元网络算法对非面部ROI进行识别的方法，SVM需要将直方图数据提升至较高的维度进行计算，而神经元网络算法需要对图像进行多次卷积卷和池化处理。因此，无论SVM还是神经元网络算法，都需要耗费较长的时间处理大量数据，造成对非面部ROI进行识别的速率较慢。
发明内容
本发明实施例提供了一种非面部ROI识别的方法及装置，能够提高对非面部ROI进行识别的速率。
第一方面，本发明实施例提供了一种非面部ROI识别的方法，包括：
从待处理的图像中确定至少一个面部关键点；
根据所述至少一个面部关键点，确定至少一个颜色样本区域；
通过所述至少一个颜色样本区域，从所述待处理的图像中提取面部图像；
对所述面部图像进行掩膜消除，获得非面部感兴趣区域ROI对应的目标图像；
根据所述至少一个面部关键点及所述目标图像，对所述待处理的图像中的所述非面部ROI进行识别。
优选地，所述通过所述至少一个颜色样本区域，从所述待处理的图像中提取面部图像，包括：
对所述至少一个颜色样本区域中的每个颜色样本区域进行颜色统计，获取该颜色样本区域对应的样本颜色，确定所述样本颜色在与该颜色样本区域相邻区域内的归一化出现频率，并将所述归一化出现频率大于预设阈值的所述相邻区域确定为该颜色样本区域对应的面部区域；
将各个所述颜色样本区域及对应的面部区域进行组合，形成组合面部区域；
从所述待处理的图像中提取位于所述组合面部区域内的图像作为所述面部图像。
优选地，所述对所述面部图像进行掩膜消除，获得非面部ROI对应的目标图像，包 括：
对所述面部图像进行掩膜消除；
根据所述至少一个面部关键点，从消除所述面部图像的待处理的图像上获得所述非面部ROI对应的目标图像。
优选地，所述根据所述至少一个面部关键点及所述目标图像，对所述待处理的图像中的所述非面部ROI进行识别，包括：
根据所述至少一个面部关键点，确定所述目标图像上干扰图像所在的区域，并将所述干扰图像消除；
根据所述目标图像上剩余图像相对于各个所述面部关键点所处位置、形状及尺寸，识别所述剩余图像是否为所述非面部ROI。
优选地，所述非面部ROI包括：舌头、牙齿、眼球及胡须中的任意一个。
第二方面，本发明实施例还一种非面部ROI识别的装置，包括：提取单元、消除单元及识别单元；
所述提取单元，用于从待处理的图像中确定至少一个面部关键点，根据所述至少一个面部关键点确定至少一个颜色样本区域，通过所述至少一个颜色样本区域从所述待处理的图像中提取面部图像；
所述消除单元，用于对所述提取单元提取出的面部图像进行掩膜消除，获得所述面部图像上非面部ROI对应的目标图像；
所述识别单元，用于根据所述提取单元确定的所述至少一个面部关键点及所述消除单元获取的所述目标图像，对所述待处理的图像中的所述非面部ROI进行识别。
优选地，所述提取单元包括：获取子单元；
所述获取子单元，对所述至少一个颜色样本区域中的每个颜色样本区域进行颜色统计，获取该颜色样本区域对应的样本颜色，确定所述样本颜色在与该颜色样本区域相邻区域内的归一化出现频率，并将所述归一化出现频率大于预设阈值的所述相邻区域确定为该颜色样本区域对应的面部区域；并将各个所述颜色样本区域及对应的面部区域进行组合，形成组合面部区域；以及从所述待处理的图像中提取位于所述组合面部区域内的图像作为所述面部图像。
优选地，
所述消除单元，用于对所述面部图像进行掩膜消除，并根据所述至少一个面部关键点，从消除所述面部图像的待处理的图像上获得所述非面部ROI相对应的目标图像。
优选地，
所述识别单元，用于根据所述至少一个面部关键点，确定所述目标图像上干扰图像所在的区域，并将所述干扰图像消除；并根据所述目标图像上剩余图像相对于各个所述面部关键点所处位置、形状及尺寸，识别所述剩余图像是否为所述非面部ROI。
优选地，
所述非面部ROI包括：舌头、牙齿、眼球及胡须中的任意一个。
第三方面，本发明实施例还提供了一种可读介质，包括执行指令，当存储控制器的处理器执行所述执行指令时，所述存储控制器执行权上述实施例提供的非面部ROI识别的方法。
第四方面，本发明实施例还提供了一种存储控制器，包括：处理器、存储器和总线；
所述存储器用于存储执行指令，所述处理器与所述存储器通过所述总线连接，当所述存储控制器运行时，所述处理器执行所述存储器存储的所述执行指令，以使所述存储控制器执行上述实施例提供的非面部ROI识别的方法。
本发明实施例提供了一种非面部ROI识别的方法、装置及可读介质和存储控制器，在待处理的图像中确定至少一个面部关键点后，根据确定出的面部关键点确定至少一个颜色样本区域，通过各个颜色样本区域从待处理图像上提取面部图像，通过对面部图像进 行掩膜消除后获得非面部ROI可能出现的目标图像，进而根据各个面部关键点及目标图像对待处理的图像中的非面部ROI进行识别。由此可见，在对非面部ROI进行识别时，仅需要对面部图像进行掩膜消除后获得可能出现非面部ROI的目标图像，进而根据各个面部关键点识别目标图像中是否包括非面部ROI，所需处理的数据量较少，从而可以提高对非面部ROI进行识别的速率。
附图简要说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明一个实施例提供的一种非面部ROI识别的方法流程图；
图2是本发明另一个实施例提供的一种非面部ROI识别的方法流程图；
图3是本发明一个实施例提供的一种非面部ROI识别的装置所在设备的示意图；
图4是本发明一个实施例提供的一种非面部ROI识别的装置示意图。
实施本发明的方式
为使本发明实施例的目的、技术方案和优点更加清楚，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本发明一部分实施例，而不是全部的实施例，基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本发明保护的范围。
如图1所示，本发明实施例提供了一种非面部ROI识别的方法，该方法可以包括以下步骤：
步骤101：从待处理的图像中确定至少一个面部关键点；
步骤102：根据所述至少一个面部关键点，确定至少一个颜色样本区域；
步骤103：通过所述至少一个颜色样本区域，从所述待处理的图像中提取面部图像；
步骤104：对所述面部图像进行掩膜消除，获得非面部感兴趣区域ROI对应的目标图像；
步骤105：根据所述至少一个面部关键点及所述目标图像，对所述待处理的图像中的所述非面部ROI进行识别。
本发明实施例提供了一种非面部ROI识别的方法，在待处理的图像中确定至少一个面部关键点后，根据确定出的面部关键点确定至少一个颜色样本区域，通过各个颜色样本区域从待处理图像上提取面部图像，通过对面部图像进行掩膜消除后获得非面部ROI可能出现的目标图像，进而根据各个面部关键点及目标图像对待处理的图像中的非面部ROI进行识别。由此可见，在对非面部ROI进行识别时，仅需要对面部图像进行掩膜消除后获得可能出现非面部ROI的目标图像，进而根据各个面部关键点识别目标图像中是否包括非面部ROI，所需处理的数据量较少，从而可以提高对非面部ROI进行识别的速率。
在本发明一个实施例中，在步骤103中通过颜色样本区域从待处理的图像中提取面部图像时，针对于每一个颜色样本区域，对该颜色样本区域进行颜色统计，获取该颜色样本区域对应的样本颜色，确定该样本颜色在与该颜色样本区域相邻的相邻区域内的归一化出现频率，并将归一化出现频率大于预设值的相邻区域确定为该颜色样本区域对应的面部区域。将各个颜色样本区域及对应的面部区域进行组合后，形成组合面部区域，进而从待处理的图像中提取位于组合面部区域内的图像作为面部图像。
例如，根据在待处理图像上确定的关键点，在待处理图像上确定2个颜色样本区域，2个颜色样本区域分别为人脸图像上两个颧骨的区域，分别为颜色样本区域1和颜色样本 区域2。对该颜色样本区域1进行颜色统计，获取颜色样本区域1内的颜色样本1，确定与颜色样本区域1相邻区域内颜色样本1的归一化出现频率，将归一化出现频率大于预设阈值80％的相邻区域确定为颜色样本区域1对应的面部区域。相应地，通过相同的方法确定颜色样本区域2对应的面部区域。将颜色样本区域1和颜色样本区域2对应的面部图像进行组合后，形成组合面部区域，将待处理图像上位于组合面部区域内的图像作为面部图像。
在待处理的图像上确定至少一个面部关键点后，根据各个面部关键点可以在面部图像上确定至少一个颜色样本区域，通过对颜色样本区域进行颜色分析，统计每一个颜色样本区域对应的样本颜色。针对每一个颜色样本区域，确定该颜色样本区域对应的颜色样本在待处理图像内的归一化出现频率，将待处理图像上与该颜色样本区域相邻且归一化出现频率大于预设阈值的区域确定为该颜色样本区域对应的面部区域。将各个颜色样本区域对应的面部区域进行组合后，形成的组合面部区域会覆盖整个面部，位于组合面部区域内的待处理图像即为面部图像。由于非面部ROI与面部图像的颜色存在较大差异，在面部图像上确定至少一个颜色样本区域，通过确定归一化出现频率的方式获取与颜色样本区域颜色相近的区域作为面部区域，将各个颜色样本区域对应的面部区域进行组合，组合面部区域内的待处理图像即为面部图像，实现面部图像的提取。在确定面部图像后边可以方便地确定待处理图像上的非面部ROI。
需要说明的是，不同颜色样本区域对应的面部区域可能会有重叠部分，在对各个颜色样本区域对应的面部区域进行组合时，通过各个面部关键点进行定位，将对应同一区域的面部区域进行重叠处理。
在本发明一个实施例中，在步骤104中对面部图像进行掩膜消除获得非面部ROI对应的目标图像时，对待处理图像上的面部图像进行眼淹没消除，在消除面部图像的待处理图像上，根据各个面部关键点获得非面部ROI对应的目标图像。
例如，非面部ROI为舌头，当获取到面部图像后，对待处理图像上的面部图像进行掩膜消除。在消除面部图像后的待处理图像上，根据之前确定出的至少一个面部关键点，对舌头可能出现的区域进行定位，获取到包括嘴唇及可能包括胡子和舌头的目标图像。
在本发明一个实施例中，在步骤105中根据面部关键点及目标图像对非面部ROI进行识别时，根据之前确定出的至少一个面部关键点，对目标图像上干扰图像所在的区域进行定位，并将干扰图像消除。针对于目标图像上消除干扰图像后剩下的剩余图像，根据剩余图像相对于各个面部关键点所处的位置，以及剩余图像的形状和相对于面部图像的尺寸，判断消除干扰图像后目标图像上的剩余图像是否为所要识别的非面部ROI。
例如，非面部ROI为舌头，所获取到的目标图像包括嘴唇等干扰图像，通过之前确定出的至少一个面部关键点，可以对嘴唇等干扰图像进行定位，在确定出目标图像上嘴唇等干扰图像后，将干扰图像消除。如果在消除嘴唇等干扰图像后目标图像还包括剩余图像，进一步根据剩余图像的位置、形状和尺寸判断剩余图像是否为舌头，如果是则确定待处理图像上包括舌头，即包括非面部ROI，否则确定待处理图像上不包括非面部ROI；如果在消除嘴唇等干扰图像后目标图像被完全消除，则确定待处理图像上不包括非面部ROI。
在本发明一个实施例中，非面部ROI可以为舌头、牙齿、眼球及胡须中的任意一种。这样，本发明实施例所提供的非面部ROI识别方法可以对多种非面部感兴趣区域进行识别，从而提高了该非面部ROI识别方法的适用性。
下面以舌头作为非面部ROI为例，对本发明实施例提供的非面部ROI识别的方法作进一步详细说明，如图2所示，该方法可以包括以下步骤：
步骤201：获取待处理图像。
在本发明一个实施例中，实时获取待处理图像。当待处理图像以视频流的形式输入时，将视频流中的每一帧图像作为一个待处理图像。
例如，在用户A与用户B进行趣味视频聊天时，通过摄像头采集用户A的视频数据，将所采集到的视频数据包括的每一帧图像作为一个待处理图像，获取第一帧待处理图像。
步骤202：判断待处理图像中是否包括面部图像，如果是，执行步骤203，否则执行步骤201。
在本发明一个实施例中，在获取到待处理图像后，对待处理图像进行人脸识别，识别待处理图像中是否包括面部图像，如果是，执行步骤203，进一步识别待处理图像上是否包括非面部ROI；如果否，则执行步骤201，获取下一个待处理图像。
例如，在获取到摄像头采集到的第一帧待处理图像后，识别该待处理图像上是否包括人脸的图像，如果是，针对该待处理图像执行步骤203；如果待处理图像上不包括人脸的图像，则执行步骤201，获取摄像头采集到的下一帧待处理图像，及第二帧待处理图像。
步骤203：在待处理图像上确定至少一个面部关键点。
在本发明一个实施例中，在确定待处理图像上面部人脸图像后，在待处理图像上确定至少一个面部关键点，其中面部关键点位于待处理图像所包括的面部图像上。
例如，在确定待处理图像A包括面部图像后，在待处理图像A上确定三个面部关键点，其中，面部关键点a为用户A面部图像上左眼所在位置，面部关键点b为用户A面部图像上右眼所在位置，面部关键点c为用户A面部图像上鼻尖所在位置。
步骤204：根据各个面部关键点确定至少一个颜色样本区域。
在本发明一个实施例中，根据步骤203中确定出的至少一个面部关键点，在待处理图像所包括的面部图像上确定至少一个颜色样本区域。
例如，在待处理图像A所包括的用户A的面部图像上确定2个颜色样本区域，其中，颜色样本区域d为用户A面部图像上左侧颧骨所在的区域，颜色样本区域e为用户A面部图像上右侧颧骨所在的区域。
步骤205：根据至少一个颜色样本区域，从待处理图像中提取面部图像。
在本发明一个实施例中，针对于每一个颜色样本区域，通过颜色分析获取该颜色样本区域对应的样本颜色，确定该样本颜色在待处理图像各个区域的归一化出现频率。将待处理图像上与该颜色样本区域相邻，且归一化出现频率大于预设阈值的区域作为该颜色样本区域对应的面部区域。将各个颜色样本区域对应的面部区域进行组合，形成组合面部区域，将待处理图像上位于组合面部区域内的图像确定为面部图像。
例如，通过对颜色样本区域d进行颜色分析，确定颜色样本区域d内图像的颜色为颜色X，将颜色X确定为颜色样本区域d的样本颜色；确定颜色X在整个待处理图像A上各个区域的归一化出现频率，将待处理图像A上与颜色样本区域d相邻且颜色X的归一化出现频率大于预设阈值80％的区域确定为颜色样本区域d对应的面部区域，比如确定出颜色样本区域d对应的面部区域为待处理图像A上用户A左半边脸、额头、下巴及鼻子的区域。相应地，通过相同的方法确定出颜色样本区域e对应的面部区域为待处理图像A上用户A右半边脸及额头的区域。将颜色样本区域d与颜色样本区域e对应的面部区域进行组合后，组合面部区域包括了待处理图像A上除用户A眼睛及嘴部之外的其他各个面部区域。从待处理图像A上提取位于组合面部区域之内的图像作为面部图像，即提取到除眼睛和嘴部之外用户A的面部图像。
步骤206：对面部图像进行掩膜消除，获得可能包括非面部ROI的目标区域。
在本发明一个实施例中，在获取到面部图像之后，对待处理图像上的面部图像进行掩膜消除，消除面部图像后获得待处理图像上可能包括非面部ROI的目标区域。
例如，在待处理图像A上对步骤205中获得的面部图像进行掩膜消除后，获得可能包括用户A舌头图像的嘴部图像，将嘴部图像确定为目标图像。
步骤207：根据各个面部关键点，对目标图像中的干扰图像进行消除，获得剩余图像。
在本发明一个实施例中，在获得目标图像后，根据步骤203中确定出的各个面部关键点，定位目标图像中存在的干扰图像，在将目标图像中的干扰图像消除后获得可能为非面部ROI的剩余图像。
例如，通过面部关键点a(左眼位置)、面部关键点b(有眼位置)和面部关键点c(鼻尖位置)对目标图像(嘴部图像)上的干扰图像(嘴唇图像)进行定位，在确定嘴唇图像 的区域后，对目标图像上的嘴唇图像进行消除，将消除嘴唇图像后的目标图像作为剩余图像。
步骤208：根据剩余图像的位置、形状及尺寸，确定剩余图像是否为非面部ROI。
在本发明一个实施例中，在获得剩余图像后，根据剩余图像与各个面部关键点的相对位置，以及剩余图像的形状及相对于面部图像的尺寸，确定剩余图像是否为非面部ROI。
例如，如果在消除干扰图像后的目标图像上还包括有图像，即存在剩余图像，确定剩余图像相对于面部关键点a、面部关键点b及面部关键点c的位置，判断剩余图像是否位于舌头可能出现的位置，如果是，则进一步判断剩余图像的形状和尺寸是否与舌头的图像相符，如果是，则确定待处理图像A上存在舌头的图像，在用户B端的显示设备上显示与舌头相对应的特效。如果在消除干扰图像后的目标图像被完全消除，或者虽然有剩余图像存在，但剩余图像的位置、形状或尺寸与舌头的位置、形状及尺寸不赌赢，则确定待处理图像A上部存在舌头的图像，相应地不会在用户B端的显示设备上显示与舌头相对应的特效。
如图3、图4所示，本发明实施例提供了一种非面部ROI识别的装置。装置实施例可以通过软件实现，也可以通过硬件或者软硬件结合的方式实现。从硬件层面而言，如图3所示，为本发明实施例提供的非面部ROI识别的装置所在设备的一种硬件结构图，除了图3所示的处理器、内存、网络接口以及非易失性存储器之外，实施例中装置所在的设备通常还可以包括其他硬件，如负责处理报文的转发芯片等等。以软件实现为例，如图4所示，作为一个逻辑意义上的装置，是通过其所在设备的CPU将非易失性存储器中对应的计算机程序指令读取到内存中运行形成的。本实施例提供的非面部ROI识别的装置，包括：提取单元401、消除单元402及识别单元403；
所述提取单元401，用于从待处理的图像中确定至少一个面部关键点，根据所述至少一个面部关键点确定至少一个颜色样本区域，通过所述至少一个颜色样本区域从所述待处理的图像中提取面部图像；
所述消除单元402，用于对所述提取单元401提取出的面部图像进行掩膜消除，获得所述面部图像上非面部ROI对应的目标图像；
所述识别单元403，用于根据所述提取单元401确定的所述至少一个面部关键点及所述消除单元402获取的所述目标图像，对所述待处理的图像中的所述非面部ROI进行识别。
在本发明一个实施例中，提取单元401包括有获取子单元；
获取子单元用于针对于每一个所述颜色样本区域，对该颜色样本区域进行颜色统计，获取该颜色样本区域对应的样本颜色，确定所述样本颜色在与该颜色样本区域相邻区域内的归一化出现频率，并将所述归一化出现频率大于预设阈值的所述相邻区域确定为该颜色样本区域对应的面部区域；并将各个所述颜色样本区域及对应的面部区域进行组合，形成组合面部区域；以及从所述待处理的图像中提取位于所述组合面部区域内的图像作为所述面部图像。
在本发明一个实施例中，所述消除单元402用于对所述面部图像进行掩膜消除，并根据所述至少一个面部关键点，从消除所述面部图像的待处理的图像上获得所述非面部ROI相对应的目标区域。
在本发明一个实施例中，所述识别单元403用于根据所述至少一个面部关键点，确定所述目标图像上干扰图像所在的区域，并将所述干扰图像消除；并根据所述目标图像上剩余图像相对于各个所述面部关键点所处位置、形状及尺寸，识别所述剩余图像是否为所述非面部ROI。
需要说明的是，上述装置内的各单元之间的信息交互、执行过程等内容，由于与本发明方法实施例基于同一构思，具体内容可参见本发明方法实施例中的叙述，此处不再赘述。
本发明实施例还提供了一种可读介质，包括执行指令，当存储控制器的处理器执行所述执行指令时，所述存储控制器执行上述实施例提供的非面部ROI识别的方法。
本发明实施例还提供了一种存储控制器，包括：处理器、存储器和总线；
所述存储器用于存储执行指令，所述处理器与所述存储器通过所述总线连接，当所述存储控制器运行时，所述处理器执行所述存储器存储的所述执行指令，以使所述存储控制器执行上述实施例提供的非面部ROI识别的方法。
本发明各个实施例提供的非面部ROI识别的方法及装置，至少具有如下有益效果：
1、在本发明实施例提供的非面部ROI识别的方法及装置中，在待处理的图像中确定至少一个面部关键点后，根据确定出的面部关键点确定至少一个颜色样本区域，通过各个颜色样本区域从待处理图像上提取面部图像，通过对面部图像进行掩膜消除后获得非面部ROI可能出现的目标图像，进而根据各个面部关键点及目标图像对待处理的图像中的非面部ROI进行识别。由此可见，在对非面部ROI进行识别时，仅需要对面部图像进行掩膜消除后获得可能出现非面部ROI的目标图像，进而根据各个面部关键点识别目标图像中是否包括非面部ROI，所需处理的数据量较少，从而可以提高对非面部ROI进行识别的速率。
2、在本发明实施例提供的非面部ROI识别的方法及装置中，在识别非面部ROI时，通过颜色分析对面部图像进行消除，进而识别剩余图像是否为非面部ROI，在识别非面部ROI过程中依赖图像的颜色，因此能够在各种环境下对非面部ROI进行识别，比如待处理图像过曝光、曝光不足等，提高了该非面部ROI识别方法及装置的适用性。
3、在本发明实施例提供的非面部ROI识别的方法及装置中，通过颜色分析、统计的方法对面部图像进行消除，进而识别待处理图像中是否存在非面部ROI，相对于现有SVM及神经元网络算法提高了识别非面部ROI的速率，保证了识别非面部ROI的时效性，从而可以将该非面部ROI识别的方法及装置应用于实时性要求较高的场合，进一步提高了该非面部ROI识别方法及装置的适用性。
4、在本发明实施例提供的非面部ROI识别的方法及装置中，非面部ROI可以为舌头、牙齿、眼球以及胡须中的任意一种，因此该非面部ROI识别的方法及装置能够满足不同用户的需求，提高了该非面部ROI识别方法及装置的通用性。
需要说明的是，在本文中，诸如第一和第二之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个······”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同因素。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本发明的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序校验码的介质。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应所述以权利要求的保护范围为准。
工业实用性
本发明实施例的非面部ROI识别方法及装置，避免了大量冗余图像信号的处理，精简了图像处理的运算量。仅需要对面部图像进行掩膜消除后获得可能出现非面部ROI的目标图像，进而根据各个面部关键点识别目标图像中是否包括非面部ROI，所需处理的数据量较少，从而可以提高对非面部ROI进行识别的速率。本发明实施例的非面部ROI识别方法及装置确定非面部ROI可以普遍应用于智能移动终端设备，改善人机交互过程的效率。

Claims (12)

  1. 一种非面部ROI识别的方法，其特征在于，包括：
    从待处理的图像中确定至少一个面部关键点；
    根据所述至少一个面部关键点，确定至少一个颜色样本区域；
    通过所述至少一个颜色样本区域，从所述待处理的图像中提取面部图像；
    对所述面部图像进行掩膜消除，获得非面部感兴趣区域ROI对应的目标图像；
    根据所述至少一个面部关键点及所述目标图像，对所述待处理的图像中的所述非面部ROI进行识别。
  2. 根据权利要求1所述非面部ROI识别的方法，其特征在于，
    所述通过所述至少一个颜色样本区域，从所述待处理的图像中提取面部图像，包括：
    对所述至少一个颜色样本区域中的每个颜色样本区域进行颜色统计，获取该颜色样本区域对应的样本颜色，确定所述样本颜色在与该颜色样本区域相邻区域内的归一化出现频率，并将所述归一化出现频率大于预设阈值的所述相邻区域确定为该颜色样本区域对应的面部区域；
    将各个所述颜色样本区域及对应的面部区域进行组合，形成组合面部区域；
    从所述待处理的图像中提取位于所述组合面部区域内的图像作为所述面部图像。
  3. 根据权利要求1或2所述非面部ROI识别的方法，其特征在于，
    所述对所述面部图像进行掩膜消除，获得非面部ROI对应的目标图像，包括：
    对所述面部图像进行掩膜消除；
    根据所述至少一个面部关键点，从消除所述面部图像的待处理的图像上获得所述非面部ROI对应的目标图像。
  4. 根据权利要求1至3任一所述非面部ROI识别的方法，其特征在于，
    所述根据所述至少一个面部关键点及所述目标图像，对所述待处理的图像中的所述非面部ROI进行识别，包括：
    根据所述至少一个面部关键点，确定所述目标图像上干扰图像所在的区域，并将所述干扰图像消除；
    根据所述目标图像上剩余图像相对于各个所述面部关键点所处位置、形状及尺寸，识别所述剩余图像是否为所述非面部ROI。
  5. 根据权利要求1至4中任一所述非面部ROI识别的方法，其特征在于，
    所述非面部ROI包括：舌头、牙齿、眼球及胡须中的任意一个。
  6. 一种非面部ROI识别的装置，其特征在于，包括：提取单元、消除单元及识别单元；
    所述提取单元，用于从待处理的图像中确定至少一个面部关键点，根据所述至少一个面部关键点确定至少一个颜色样本区域，通过所述至少一个颜色样本区域从所述待处理的图像中提取面部图像；
    所述消除单元，用于对所述提取单元提取出的面部图像进行掩膜消除，获得所述面部图像上非面部ROI对应的目标图像；
    所述识别单元，用于根据所述提取单元确定的所述至少一个面部关键点及所述消除单元获取的所述目标图像，对所述待处理的图像中的所述非面部ROI进行识别。
  7. 根据权利要求6所述非面部ROI识别的装置，其特征在于，
    所述提取单元包括：获取子单元；
    所述获取子单元，对所述至少一个颜色样本区域中的每个颜色样本区域进行颜色统计，获取该颜色样本区域对应的样本颜色，确定所述样本颜色在与该颜色样本区域相邻区域内的归一化出现频率，并将所述归一化出现频率大于预设阈值的所述相邻区域确定 为该颜色样本区域对应的面部区域；并将各个所述颜色样本区域及对应的面部区域进行组合，形成组合面部区域；以及从所述待处理的图像中提取位于所述组合面部区域内的图像作为所述面部图像。
  8. 根据权利要求6所述非面部ROI识别的装置，其特征在于，
    所述消除单元，用于对所述面部图像进行掩膜消除，并根据所述至少一个面部关键点，从消除所述面部图像的待处理的图像上获得所述非面部ROI相对应的目标图像。
  9. 根据权利要求6至8任一所述非面部ROI识别的装置，其特征在于，
    所述识别单元，用于根据所述至少一个面部关键点，确定所述目标图像上干扰图像所在的区域，并将所述干扰图像消除；并根据所述目标图像上剩余图像相对于各个所述面部关键点所处位置、形状及尺寸，识别所述剩余图像是否为所述非面部ROI。
  10. 根据权利要求6至9任一所述非面部ROI识别的装置，其特征在于，
    所述非面部ROI包括：舌头、牙齿、眼球及胡须中的任意一个。
  11. 一种可读介质，其特征在于，包括执行指令，当存储控制器的处理器执行所述执行指令时，所述存储控制器执行权利要求1至5中任一所述的方法。
  12. 一种存储控制器，其特征在于，包括：处理器、存储器和总线；
    所述存储器用于存储执行指令，所述处理器与所述存储器通过所述总线连接，当所述存储控制器运行时，所述处理器执行所述存储器存储的所述执行指令，以使所述存储控制器执行权利要求1至5中任一所述的方法。
PCT/CN2018/075675 2017-02-20 2018-02-07 一种非面部roi识别方法及装置 WO2018149350A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710090091.6A CN106778751B (zh) 2017-02-20 2017-02-20 一种非面部roi识别方法及装置
CN201710090091.6 2017-02-20

Publications (1)

Publication Number Publication Date
WO2018149350A1 true WO2018149350A1 (zh) 2018-08-23

Family

ID=58957700

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/075675 WO2018149350A1 (zh) 2017-02-20 2018-02-07 一种非面部roi识别方法及装置

Country Status (2)

Country Link
CN (1) CN106778751B (zh)
WO (1) WO2018149350A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106778751B (zh) * 2017-02-20 2020-08-21 迈吉客科技（北京）有限公司 一种非面部roi识别方法及装置
CN108197593B (zh) * 2018-01-23 2022-02-18 深圳极视角科技有限公司 基于三点定位方法的多尺寸人脸表情识别方法及装置
CN110210319A (zh) * 2019-05-07 2019-09-06 平安科技（深圳）有限公司 计算机设备、舌体照片体质识别装置及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006103241A2 (fr) * 2005-03-31 2006-10-05 France Telecom Système et procédé de localisation de points d'intérêt dans une image d'objet mettant en œuvre un réseau de neurones
CN102663682A (zh) * 2012-03-13 2012-09-12 华中科技大学 基于感兴趣区域的自适应图像增强方法
CN105184249A (zh) * 2015-08-28 2015-12-23 百度在线网络技术（北京）有限公司 用于人脸图像处理的方法和装置
CN105787878A (zh) * 2016-02-25 2016-07-20 杭州格像科技有限公司 一种美颜处理方法及装置
CN106022214A (zh) * 2016-05-04 2016-10-12 南京工程学院 非约束环境下的有效人脸特征提取方法
CN106295566A (zh) * 2016-08-10 2017-01-04 北京小米移动软件有限公司 人脸表情识别方法及装置
CN106778751A (zh) * 2017-02-20 2017-05-31 迈吉客科技（北京）有限公司 一种非面部roi识别方法及装置

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006103241A2 (fr) * 2005-03-31 2006-10-05 France Telecom Système et procédé de localisation de points d'intérêt dans une image d'objet mettant en œuvre un réseau de neurones
CN102663682A (zh) * 2012-03-13 2012-09-12 华中科技大学 基于感兴趣区域的自适应图像增强方法
CN105184249A (zh) * 2015-08-28 2015-12-23 百度在线网络技术（北京）有限公司 用于人脸图像处理的方法和装置
CN105787878A (zh) * 2016-02-25 2016-07-20 杭州格像科技有限公司 一种美颜处理方法及装置
CN106022214A (zh) * 2016-05-04 2016-10-12 南京工程学院 非约束环境下的有效人脸特征提取方法
CN106295566A (zh) * 2016-08-10 2017-01-04 北京小米移动软件有限公司 人脸表情识别方法及装置
CN106778751A (zh) * 2017-02-20 2017-05-31 迈吉客科技（北京）有限公司 一种非面部roi识别方法及装置

Also Published As

Publication number Publication date
CN106778751B (zh) 2020-08-21
CN106778751A (zh) 2017-05-31

Similar Documents

Publication Publication Date Title
JP7046625B2 (ja) 顔認証方法及び装置
US11227158B2 (en) Detailed eye shape model for robust biometric applications
JP7229174B2 (ja) 人識別システム及び方法
CN109952594B (zh) 图像处理方法、装置、终端及存储介质
WO2018041237A1 (zh) 人脸验证的方法、装置以及存储介质
KR101198322B1 (ko) 얼굴 표정 인식 방법 및 시스템
WO2016089529A1 (en) Technologies for learning body part geometry for use in biometric authentication
WO2021218293A1 (zh) 图像处理方法、装置、电子设备和存储介质
WO2018149350A1 (zh) 一种非面部roi识别方法及装置
CN111598038B (zh) 脸部特征点检测方法、装置、设备及存储介质
CN110532965B (zh) 年龄识别方法、存储介质及电子设备
JPWO2019003973A1 (ja) 顔認証装置、顔認証方法およびプログラム
JP6784261B2 (ja) 情報処理装置、画像処理システム、画像処理方法及びプログラム
WO2021139167A1 (zh) 人脸识别方法、装置、电子设备及计算机可读存储介质
US11727784B2 (en) Mask wearing status alarming method, mobile device and computer readable storage medium
Cornejo et al. Facial expression recognition with occlusions based on geometric representation
CN111902821A (zh) 检测动作以阻止识别
US10803677B2 (en) Method and system of automated facial morphing for eyebrow hair and face color detection
CN113705466A (zh) 用于遮挡场景、尤其高仿遮挡下的人脸五官遮挡检测方法
Teng et al. Facial expressions recognition based on convolutional neural networks for mobile virtual reality
JP2017204280A (ja) ビデオフレームを選択する方法、システム及び装置
WO2022111271A1 (zh) 服装规范化检测方法及装置
AU2018420579A1 (en) Systems and methods for shaking action recognition based on facial feature points
Hsu et al. Extraction of visual facial features for health management
JP2022119467A (ja) 脈波推定装置、脈波推定方法および脈波推定プログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18753634

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS (EPO FORM 1205A DATED 04.12.2019)

122 Ep: pct application non-entry in european phase

Ref document number: 18753634

Country of ref document: EP

Kind code of ref document: A1