WO2020259129A1 - 图像处理的方法、装置、设备及计算机可读存储介质 - Google Patents

图像处理的方法、装置、设备及计算机可读存储介质 Download PDF

Info

Publication number
WO2020259129A1
WO2020259129A1 PCT/CN2020/091046 CN2020091046W WO2020259129A1 WO 2020259129 A1 WO2020259129 A1 WO 2020259129A1 CN 2020091046 W CN2020091046 W CN 2020091046W WO 2020259129 A1 WO2020259129 A1 WO 2020259129A1
Authority
WO
WIPO (PCT)
Prior art keywords
area
original image
image
key points
region
Prior art date
Application number
PCT/CN2020/091046
Other languages
English (en)
French (fr)
Inventor
卢江虎
姚聪
Original Assignee
北京迈格威科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京迈格威科技有限公司 filed Critical 北京迈格威科技有限公司
Priority to US17/612,750 priority Critical patent/US20220245777A1/en
Publication of WO2020259129A1 publication Critical patent/WO2020259129A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/04—Context-preserving transformations, e.g. by using an importance map
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/48—Extraction of image or video features by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/165—Detection; Localisation; Normalisation using facial parts and geometric relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • This application relates to the field of computer technology. Specifically, this application relates to an image processing method, device, device, and computer-readable storage medium.
  • this application proposes an image processing method, device, equipment and computer-readable storage medium to solve how to quickly realize that the target image is similar to the original image in all regions and as a whole, thereby There is a problem of beautification effect.
  • this application provides an image processing method, including:
  • the material area images that match each area of the original image are spliced to obtain the target image corresponding to the original image.
  • this application provides an image processing device, including:
  • the first processing module is used to obtain multiple key points in each area of the original image, and each area corresponds to a different body part in the portrait;
  • the second processing module is configured to determine the shape context histogram feature of each area of the original image according to multiple key points of each area;
  • the third processing module is configured to determine multiple cost matrices for each area according to the shape context histogram feature of each area of the original image and the preset shape context histogram feature of each material area image of the corresponding area;
  • the fourth processing module is configured to filter out the material area images that match each area of the original image from the material area images according to the multiple cost matrices for each area;
  • the fifth processing module is used for stitching together material area images that match each area of the original image to obtain a target image corresponding to the original image.
  • this application provides an electronic device, including: a processor, a memory, and a bus;
  • Bus used to connect the processor and memory
  • Memory used to store operation instructions
  • the processor is configured to execute the image processing method of the first aspect of the present application by invoking operation instructions.
  • this application provides a computer-readable storage medium storing a computer program, and the computer program is used to execute the image processing method of the first aspect of this application.
  • the context histogram feature determines multiple cost matrices for each area; selects each material area image according to the multiple cost matrices for each area A material area image that matches each area of the original image; and the material area images that match each area of the original image are spliced to obtain a target image corresponding to the original image.
  • each area of the original image is quickly matched to the corresponding material area image, and the matched material area images are stitched, so that the target image and the original image are very similar in whole and in each area, resulting in beautification
  • the effect of this has significantly improved the user experience.
  • FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of this application.
  • FIG. 2 is a schematic flowchart of another image processing method provided by an embodiment of the application.
  • FIG. 3 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the application.
  • FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the application.
  • An embodiment of the present application provides an image processing method.
  • a schematic flowchart of the method is shown in FIG. 1, and the method includes:
  • S101 Acquire multiple key points in each area of the original image, and each area corresponds to each different body part in the portrait.
  • the area may be various facial organ areas of the original image, such as eye area, eyebrow area, eye area, or hair area, and multiple key points are located on the contour of the corresponding area.
  • acquiring multiple key points of each area of the original image includes:
  • the multiple dense key points of each facial organ region of the original image are sampled to obtain multiple key points of each facial organ region of the sampled original image.
  • the dense key point is an upgraded version of the ordinary key point.
  • a person’s face has 81 or 106 ordinary key points, a person’s face has 1,000 dense key points, and facial organs have varying numbers of dense keys. Points, for example, there are 273 dense key points on the contour of the face, 64 dense key points on the contour of the eyebrow area, and 63 dense key points on the contour of the eye area.
  • the key points are effectively sampled to a preset number, which is the first value.
  • the shape context histogram feature of each material area image is stored in the high-performance key-value database Redis for query use.
  • the key value types supported by Redis are: String character type, map hash type, list list type, set collection type and sortedset ordered collection type.
  • the contour of the eyebrow area there are 64 dense key points on the contour of the eyebrow area, but there are only 30 common key points on the contour of the eyebrow area in the high-performance key-value database Redis. Therefore, 64 dense key points on the contour of the eyebrow area need to be checked.
  • the key points are sampled to obtain 30 dense key points, so that the number of the 30 dense key points on the contour of the eyebrow region is the same as the number of 30 ordinary key points on the contour of the eyebrow region in the high-performance key-value database Redis.
  • the first value is 30.
  • acquiring multiple key points of each area of the original image includes:
  • the contours of the hair region of the original image are sampled to obtain multiple key points of the hair region of the original image.
  • S102 Determine the shape context histogram feature of each area of the original image according to multiple key points of each area.
  • the preset shape context vector extractor extracts a vector for a key point in the region, this vector represents the key point, and a shape context histogram including multiple vectors extracted by the preset shape context vector extractor Figure features to show.
  • the shape context histogram feature is a feature description method based on the shape contour. It can well reflect the distribution of sampling points on the contour by using the histogram to describe the shape feature in the logarithmic polar coordinate system.
  • S103 Determine multiple cost matrices for each region according to the shape context histogram feature of each region of the original image and the preset shape context histogram feature of each material region image of the corresponding region.
  • the material area image is a cartoon material designed for each organ of the human body, such as a cartoon material designed for eyebrows, a nose, and a mouth, or a cartoon material designed for human hair.
  • the preset shape context histogram features of each material area image are stored in the high-performance key-value database Redis, and the high-performance key-value database Redis is queried and used for each preset material area image.
  • One area of the original image is eyebrows, and the preset cartoon material in the corresponding area is also eyebrows.
  • the chi-square distance can be obtained between each key point in the facial organ area and each key point in the material area, so that a cost matrix can be formed, and each element of the cost matrix represents a certain key in the facial organ area
  • the shape context histogram feature of each facial organ region of the original image is calculated to obtain multiple cost matrices for each facial organ area of the original image and multiple cost matrices for the hair area of the original image.
  • the preset material area images include each facial organ material image and hair material image .
  • Each element of the cost matrix represents the chi-square distance between a key point of a region and a key point of a material region image of the corresponding region.
  • S104 According to multiple cost matrices for each area, filter out material area images that match each area of the original image from each material area image.
  • each key point of any area is matched to any preset material area image with the smallest chi-square distance from each key point
  • Add the minimum chi-square distance corresponding to all the key points in any area as the cost value between any area and any material area image to obtain the difference between any area and the preset N material area images N cost values between N cost values, where N is a positive integer, and the material area image corresponding to the smallest cost value among the N cost values is taken as the material area image that matches any area.
  • N first cost values between any facial organ and the preset N facial organ material region images are obtained through the Hungarian algorithm, N is a positive integer, and the facial organ material region image corresponding to the smallest first cost value among the N first cost values is taken as the material region image matching any facial organ.
  • the Hungarian algorithm is used to obtain M second cost values between the hair region of the original image and the preset M hair material region images, where M is positive Integer, the hair material area image corresponding to the smallest second cost value among the M second cost values is used as the material area image that matches the hair of the original image.
  • the material area images that match each area of the original image are spliced according to the position of the corresponding area in the original image to obtain the target image corresponding to the original image.
  • the manner of determining the shape context histogram feature of the preset image of each material area includes:
  • the multiple key points of each material area image are input into a preset shape context vector extractor for shape context histogram feature extraction, and the preset shape context histogram feature of each material area image is obtained.
  • the alpha channel is an 8-bit gray-scale channel, which uses 256-level gray to record the transparency information in the image, defining transparent, opaque and semi-transparent areas, where white means opaque, black means transparent, and gray Represents translucent.
  • the material is cartoon material
  • the material area images matching each area of the original image are spliced to obtain the target image corresponding to the original image, including:
  • the cartoon material area images matching each area of the original image are spliced to obtain the cartoon image corresponding to the original image.
  • a lot of cartoon materials are designed for each organ of the human body, such as eyebrows, nose, mouth, etc., as well as hair.
  • search for the most similar cartoon material from the material library and stitch these most similar cartoon materials together to produce a cartoonized character image.
  • the cartoonized character image is similar to that of a real person.
  • the images have a greater degree of similarity.
  • multiple key points of each area of the original image are obtained, and each area corresponds to each different body part in the portrait; according to the multiple key points of each area, the shape context histogram feature of each area of the original image is determined; The shape context histogram feature of each area of the original image and the preset shape context histogram feature of each material area image of the corresponding area determine multiple cost matrices for each area; according to the multiple cost matrices for each area, from each From the material area image, the material area image that matches each area of the original image is selected; the material area images that match each area of the original image are spliced to obtain the target image corresponding to the original image.
  • each area of the original image is quickly matched to the corresponding material area image, and the matched material area images are stitched, so that the target image and the original image are very similar in whole and in each area, resulting in beautification
  • the effect of this has significantly improved the user experience.
  • An embodiment of the present application provides another image processing method.
  • a schematic flowchart of the method is shown in FIG. 2, and the method includes:
  • the original image is a real person image.
  • Dense key points are an upgraded version of common key points.
  • a person’s facial organs have 81 or 106 common key points, a person’s facial organs have 1,000 dense key points, and facial organs have varying numbers of dense key points. For example, there are 273 dense key points on the contour of the face in the facial organs, 64 dense key points on the contour of the eyebrow area in the facial organs, and 63 dense key points on the contour of the eye area in the facial organs.
  • S202 Classify the dense key points of the facial organs of the real person image to obtain multiple dense key points of each facial organ region of the real person image.
  • the dense key points of a person’s facial organs are classified to obtain the dense key points of each facial organ region in the facial organs, for example, the dense key points on the contour of the left eyebrow region, and compare the dense key points of the left eyebrow region.
  • the dense key points on the contour are effectively sampled to a preset number, which is the first value.
  • the shape context histogram feature of the cartoon material area image is stored in the high-performance key-value database Redis for query use.
  • the key value types supported by Redis are: String character type, map hash type, list list type, set collection type and sortedset ordered collection type.
  • the contour of the eyebrow area there are 64 dense key points on the contour of the eyebrow area, but there are only 30 common key points on the contour of the eyebrow area in the high-performance key-value database Redis. Therefore, 64 dense key points on the contour of the eyebrow area need to be checked.
  • the key points are sampled to obtain 30 dense key points, so that the number of the 30 dense key points on the contour of the eyebrow region is the same as the number of 30 ordinary key points on the contour of the eyebrow region in the high-performance key-value database Redis.
  • the first value is 30.
  • S204 Determine the contour of the hair region of the real person image according to the grayscale image of the hair region of the real person image, and sample the contour of the hair region of the real person image to obtain multiple key points of the hair region of the real person image.
  • S205 Obtain the shape context histogram feature of each facial organ area of the real person image and the hair of the real person image based on the multiple dense key points of each facial organ area of the real person image and the multiple area key points of the hair area of the real person image The shape context histogram feature of the region.
  • S206 Call the shape context histogram feature of the cartoon material area image from Redis, and calculate through the chi-square distance to obtain multiple cost matrices for each facial organ area of the real person image and multiple cost matrices for the hair area of the real person image .
  • the shape context histogram feature of the cartoon material area image from Redis, according to the shape context histogram of each facial organ area of the real person image Features, the shape context histogram feature of the hair area of the real person image and the shape context histogram feature of each cartoon material area image of the corresponding area, and use the chi-square distance formula (1) to calculate the corresponding COST cost matrix, the formula (1) is as follows Shown:
  • x in formula (1) is the key point represented by the vector in the shape context histogram feature of the cartoon material area image
  • y is the shape context histogram feature of each facial organ area of the real person image or the hair area of the real person image
  • the chi-square distance between each key point of the facial organ area and each key point of the cartoon material area is obtained by formula (1), so that the cost matrix can be formed, and each element of the cost matrix represents a certain part of the facial organ area
  • the chi-square distance between a key point and a key point in the cartoon material area. Chi-square distance is used to measure the difference between a certain key point in the facial organ area and a certain key point in the cartoon material area.
  • S207 According to multiple cost matrices for each area of the real person image, filter out cartoon material area images that match each area of the real person image from the cartoon material area images.
  • the Hungarian algorithm is used to obtain N first cost values between any facial organ and the preset N facial organ cartoon region material images
  • the value of N is 10
  • the facial organ cartoon material area image corresponding to the smallest first cost value among the ten first cost values is taken as the cartoon material area image matching any facial organ.
  • Different facial organs can correspond to different preset numbers of materials, and the number of first cost values obtained is also different.
  • the minimum chi-square distance of all key points is added as the first cost value of the facial organ and the facial organ material.
  • the Hungarian algorithm is used to obtain M second cost values between the hair area of the real person image and the preset M hair material area images, for example, M The value is 10, and the hair cartoon material area image corresponding to the smallest second cost value among the 10 second cost values is used as the cartoon material area image that matches the hair of the real person image.
  • the cartoon material area images matching each area of the real person image are spliced to obtain a cartoon image corresponding to the real person image.
  • each area of the original image of the real person is matched to the corresponding cartoon material area image, and the matched cartoon material area images are spliced to obtain the cartoon image corresponding to the original image of the real person.
  • the cartoon image is in the original image of the real person. The whole and each area are very similar, which produces a beautifying effect and significantly improves the user experience.
  • an embodiment of the present application also provides an image processing device.
  • the schematic diagram of the device is shown in FIG. 3.
  • the image processing device 30 includes a first processing module 301, a second processing module 302, The third processing module 303, the fourth processing module 304, and the fifth processing module 305.
  • the first processing module 301 is used to obtain multiple key points in each area of the original image, each area corresponding to each different body part in the portrait;
  • the second processing module 302 is configured to determine the shape context histogram feature of each area of the original image according to multiple key points of each area;
  • the third processing module 303 is configured to determine multiple cost matrices for each area according to the shape context histogram feature of each area of the original image and the preset shape context histogram feature of each material area image of the corresponding area;
  • the fourth processing module 304 is configured to filter out the material area images that match each area of the original image from the material area images according to multiple cost matrices for each area;
  • the fifth processing module 305 is used to splice the material area images matching each area of the original image to obtain a target image corresponding to the original image.
  • the first processing module 301 is specifically configured to obtain multiple dense key points of the original image; classify the multiple dense key points of the original image, Obtain multiple dense key points of each facial organ area of the original image; sample multiple dense key points of each facial organ area of the original image to obtain multiple keys of each facial organ area of the sampled original image point.
  • the first processing module 301 is specifically configured to obtain the gray image of the hair area of the original image; determine the original image according to the gray image of the hair area of the original image. The contour of the hair area of the image; sampling the contour of the hair area of the original image to obtain multiple key points of the hair area of the original image.
  • the second processing module 302 is specifically configured to input multiple key points of each facial organ region of the original image and multiple region key points of the hair region of the original image into a preset shape context vector extractor for processing.
  • the shape context histogram feature is extracted to obtain the shape context histogram feature of each facial organ region of the original image and the shape context histogram feature of the hair region of the original image.
  • the third processing module 303 is specifically configured to use the shape context histogram feature of each facial organ region of the original image, the shape context histogram feature of the hair region of the original image, and preset material regions of the corresponding region.
  • the shape context histogram feature of the image is calculated by chi-square distance to obtain multiple cost matrices for each facial organ area of the original image and multiple cost matrices for the hair area of the original image.
  • the preset images of each material area include For each facial organ material image and hair material image, each element of the cost matrix represents the chi-square distance between a key point of the area and a key point of a material area image of the corresponding area.
  • the fourth processing module 304 is specifically configured to match each key point in any area to the preset AND in any material area image according to the N cost matrices corresponding to any area of the original image. For the key point with the smallest chi-square distance of each key point, add the minimum chi-square distance corresponding to all key points in any area as the cost value between any area and any material area image, and get any area and N preset cost values between N material area images, where N is a positive integer, and the material area image corresponding to the smallest cost value among the N cost values is taken as the material area image matching any area.
  • the manner of determining the shape context histogram feature of the preset image of each material area includes:
  • the multiple key points of each material area image are input into a preset shape context vector extractor for shape context histogram feature extraction, and the preset shape context histogram feature of each material area image is obtained.
  • the material is cartoon material
  • the fifth processing module 305 is specifically configured to stitch together cartoon material area images that match each area of the original image to obtain a cartoon image corresponding to the original image.
  • the context histogram feature determines multiple cost matrices for each area; selects each material area image according to the multiple cost matrices for each area A material area image that matches each area of the original image; and the material area images that match each area of the original image are spliced to obtain a target image corresponding to the original image.
  • each area of the original image is quickly matched to the corresponding material area image, and the matched material area images are stitched, so that the target image and the original image are very similar in whole and in each area, resulting in beautification
  • the effect of this has significantly improved the user experience.
  • an embodiment of the present application also provides an electronic device.
  • a schematic structural diagram of the electronic device is shown in FIG. 4.
  • the electronic device 7000 includes at least one processor 7001, a memory 7002, and a bus 7003.
  • the memory 7001 is electrically connected to the storage 7002; the memory 7002 is configured to store at least one computer-executable instruction, and the processor 7001 is configured to execute the at least one computer-executable instruction, so as to execute any one in the first embodiment of the present application. Steps of any image processing method provided by an embodiment or any optional implementation.
  • the processor 7001 may be an FPGA (Field-Programmable Gate Array) or other devices with logic processing capabilities, such as MCU (Microcontroller Unit), CPU (Central Process Unit, Central Processing Unit) ).
  • FPGA Field-Programmable Gate Array
  • MCU Microcontroller Unit
  • CPU Central Process Unit
  • Central Processing Unit Central Processing Unit
  • the context histogram feature determines multiple cost matrices for each area; selects each material area image according to the multiple cost matrices for each area A material area image that matches each area of the original image; and the material area images that match each area of the original image are spliced to obtain a target image corresponding to the original image.
  • each area of the original image is quickly matched to the corresponding material area image, and the matched material area images are stitched, so that the target image and the original image are very similar in whole and in each area, resulting in beautification
  • the effect of this has significantly improved the user experience.
  • the embodiment of the present application also provides a computer-readable storage medium, such as the memory 7002 in FIG. 4, in which a computer program 7002a is stored, which is used to implement the implementation of the present application when executed by a processor.
  • a computer-readable storage medium such as the memory 7002 in FIG. 4, in which a computer program 7002a is stored, which is used to implement the implementation of the present application when executed by a processor.
  • the computer-readable storage medium includes but is not limited to any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM ( Random Access Memory), EPROM (Erasable Programmable Read-Only Memory, Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), Flash memory, Magnetic Card or light card. That is, a readable storage medium includes any medium that stores or transmits information in a readable form by a device (for example, a computer).
  • the context histogram feature determines multiple cost matrices for each area; selects each material area image according to the multiple cost matrices for each area A material area image that matches each area of the original image; and the material area images that match each area of the original image are spliced to obtain a target image corresponding to the original image.
  • each area of the original image is quickly matched to the corresponding material area image, and the matched material area images are stitched, so that the target image and the original image are very similar in whole and in each area, resulting in beautification
  • the effect of this has significantly improved the user experience.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Geometry (AREA)
  • Image Analysis (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本申请实施例提供了一种图像处理的方法、装置、设备及计算机可读存储介质，该方法包括：获取原始图像各区域的多个关键点，各区域对应人像中不同的各个身体部位；根据各区域的多个关键点，确定原始图像各区域的形状上下文直方图特征；根据原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确定针对各个区域的多个成本矩阵；根据针对各个区域的多个成本矩阵，从各素材区域图像中筛选出与所述原始图像每一区域相匹配的素材区域图像；将与原始图像每一区域相匹配的素材区域图像进行拼接，得到原始图像对应的目标图像。该方法快速实现了目标图像与原始图像在整体和各区域上都很相似，产生了美化效果。

Description

图像处理的方法、装置、设备及计算机可读存储介质
本申请要求在2019年6月27日提交中国专利局、申请号为201910570341.5、发明名称为“图像处理的方法、装置、设备及计算机可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，具体而言，本申请涉及一种图像处理的方法、装置、设备及计算机可读存储介质。
背景技术
随着直播行业的火爆和人工智能技术的兴起，各种美颜算法实时运行在直播镜头之前，给人一种美感。人们有在网络世界构建另一个完美自己的需求，希望通过已有的素材，快速生成与真人图像(原始图像)的各区域和真人图像的整体上都很相似的素材图像(目标图像)，要美且像本人。目前主要的美颜手段都是对人脸进行磨皮，或者加一些特效，美化效果差。
发明内容
本申请针对现有的方式的缺点，提出一种图像处理的方法、装置、设备及计算机可读存储介质，用以解决如何快速实现目标图像与原始图像在各区域和整体上都很相似，从而产生了美化效果的问题。
第一方面，本申请提供了一种图像处理的方法，包括：
获取原始图像各区域的多个关键点，各区域对应人像中不同的各个身体部位；
根据各区域的多个关键点，确定原始图像各区域的形状上下文直方图特征；
根据原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确定针对各个区域的多个成本矩阵；
根据针对各个区域的多个成本矩阵，从各素材区域图像中筛选出与原始图像每一区域相匹配的素材区域图像；
将与原始图像每一区域相匹配的素材区域图像进行拼接，得到原始图像对应的目标图像。
第二方面，本申请提供了一种图像处理的装置，包括：
第一处理模块，用于获取原始图像各区域的多个关键点，各区域对应人像中不同的各个身体部位；
第二处理模块，用于根据所述各区域的多个关键点，确定所述原始图像各区域的形状上下文直方图特征；
第三处理模块，用于根据所述原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确定针对各个区域的多个成本矩阵；
第四处理模块，用于根据所述针对各个区域的多个成本矩阵，从所述各素材区域图像中筛选出与所述原始图像每一区域相匹配的素材区域图像；
第五处理模块，用于将与所述原始图像每一区域相匹配的素材区域图像进行拼接，得到所述原始图像对应的目标图像。
第三方面，本申请提供了一种电子设备，包括：处理器、存储器和总线；
总线，用于连接处理器和存储器；
存储器，用于存储操作指令；
处理器，用于通过调用操作指令，执行本申请第一方面的图像处理的方法。
第四方面，本申请提供了一种计算机可读存储介质，存储有计算机程序，计算机程序被用于执行本申请第一方面的图像处理的方法。
本申请实施例提供的技术方案，至少具有如下有益效果：
获取原始图像各区域的多个关键点，各区域对应人像中不同的各个身体部位；根据各区域的多个关键点，确定原始图像各区域的形状上下文直方图特征；根据原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确定针对各个区域的多个成本矩阵；根据针对各个区域的多个成本矩阵，从各素材区域图像中筛选出与所述原始图像每一区域相匹配的素材区域图像；将与原始图像每一区域相匹配的素材区域图像进行拼接，得到原始图像对应的目标图像。如此，实现了快速的为原始图像各区域匹配到对应的素材区域图像，将匹配到的素材区域图像进行拼接，从而得到的目标图像与原始图像在整体和各区域上都很相似，产生了美化的效果，显著地提升了用户体验。
本申请附加的方面和优点将在下面的描述中部分给出，这些将从下面的描述中变得明显，或通过本申请的实践了解到。
上述说明仅是本发明技术方案的概述，为了能够更清楚了解本发明的技术手段，而可依照说明书的内容予以实施，并且为了让本发明的上述和其它目的、特征和优点能够更明显易懂，以下特举本发明的具体实施方式。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对本申请实施例描述中所需要使用的附图作简单地介绍。
图1为本申请实施例提供的一种图像处理的方法的流程示意图；
图2为本申请实施例提供的另一种图像处理的方法的流程示意图；
图3为本申请实施例提供的一种图像处理的装置的结构示意图；
图4为本申请实施例提供的一种电子设备的结构示意图。
具体实施例
下面详细描述本申请的实施例，所述实施例的示例在附图中示出，其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的，仅用于解释本申请，而不能解释为对本发明的限制。
本技术领域技术人员可以理解，除非特意声明，这里使用的单数形式“一”、“一个”、“所述”和“该”也可包括复数形式。应该进一步理解的是，本申请的说明书中使用的措辞“包括”是指存在所述特征、整数、步骤、操作、元件和/或组件，但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元件、组件和/或它们的组。应该理解，当我们称元件被“连接”或“耦接”到另一元件时，它可以直接连接或耦接到其他元件，或者也可以存在中间元件。此外，这里使用的“连接”或“耦接”可以包括无线连接或无线耦接。这里使用的措辞“和/或”包括一个或更多个相关联的列出项的全部或任一单元和全部组合。
本技术领域技术人员可以理解，除非另外定义，这里使用的所有术语(包括技术术语和科学术语)，具有与本发明所属领域中的普通技术人员的一般理解相同的意义。还应该理解的是，诸如通用字典中定义的那些术语，应该被理解为具有与现有技术的上下文中的意义一致的意义，并且除非像这里一样被特定定义，否则不会用理想化或过于正式的含义来解释。
下面以具体地实施例对本申请的技术方案以及本申请的技术方案如何解决上述技术问题进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例中不再赘述。下面将结合附图，对本申请的实施例进行描述。
实施例一
本申请实施例中提供了一种图像处理的方法，该方法的流程示意图如图1所示，该方法包括：
S101，获取原始图像各区域的多个关键点，各区域对应人像中不同的各个身体部位。
可选地，区域可以为原始图像的各面部器官区域，例如眼部区域、眉毛区域、眼睛区域，还可以是头发区域，多个关键点位于对应区域的轮廓上。
可选地，当原始图像各区域为原始图像的各面部器官区域时，获取原始图像各区域的多个关键点，包括：
获取原始图像的多个稠密关键点；
将原始图像的多个稠密关键点进行分类，得到原始图像的每一面部器官区域的多个稠密关键点；
对原始图像的每一面部器官区域的多个稠密关键点进行采样，得到采样后的原始图像的每一面部器官区域的多个关键点。使用稠密关键点提取器官的轮廓，并进行有效采样，得到比较稀疏但是完整的器官轮廓，降低后续匹配的时间复杂度。
可选地，稠密关键点是普通关键点的升级版，一个人的面部有81个或106个普通关键点，一个人的面部有1000个稠密关键点，面部器官都有数量不等的稠密关键点，例如，脸的轮廓上有273个稠密关键点，眉毛区域的轮廓上有64个稠密关键点，眼睛区域的轮廓上有63个稠密关键点。
可选地，对一个人的面部的稠密关键点进行分类，得到面部的每个局部器官的稠密关键点，例如左眉区域的轮廓上的稠密关键点，并对左眉区域的轮廓上的稠密关键点进行有效采样到一个预设的数量，该数量为第一数值。
可选地，将各素材区域图像的形状上下文直方图特征存入到高性能键值数据库Redis中作查询使用。Redis支持的键值类型有：String字符类型、map散列类型、list列表类型、set集合类型和sortedset有序集合类型。
可选地，眉毛区域的轮廓上有64个稠密关键点，但是高性能键值数据库Redis中的眉毛区域的轮廓上只有30个普通关键点，因此，需要对眉毛区域的轮廓上的64个稠密关键点进行采样，得到30个稠密关键点，这样眉毛区域的轮廓上的30个稠密关键点的数量与高性能键值数据库Redis中的眉毛区域的轮廓上的30个普通关键点的数量相同， 第一数值取值为30。
可选地，当原始图像的区域为原始图像的头发区域时，获取原始图像各区域的多个关键点，包括：
获取原始图像的头发区域的灰度图；
根据原始图像的头发区域的灰度图，确定原始图像的头发区域的轮廓；
对原始图像的头发区域的轮廓进行采样，得到原始图像的头发区域的多个区域关键点。
S102，根据各区域的多个关键点，确定原始图像各区域的形状上下文直方图特征。
可选地，将原始图像的每一面部器官区域的多个关键点和原始图像的头发区域的多个区域关键点，输入预设的形状上下文向量提取器进行形状上下文直方图特征提取，得到原始图像的每一面部器官区域的形状上下文直方图特征和原始图像的头发区域的形状上下文直方图特征。
可选地，预设的形状上下文向量提取器为区域中一个关键点提取一个向量，这个向量表示这个关键点，一个区域由预设的形状上下文向量提取器提取的包括多个向量的形状上下文直方图特征来表示。形状上下文直方图特征是采用一种基于形状轮廓的特征描述方法,其在对数极坐标系下利用直方图描述形状特征能够很好地反映轮廓上采样点的分布情况。
S103，根据原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确定针对各个区域的多个成本矩阵。
可选地，素材区域图像是针对人身体的每个器官设计的卡通素材，例如眉毛、鼻子、嘴巴的卡通素材，或针对人的头发设计的卡通素材。预设的各素材区域图像的形状上下文直方图特征存放在高性能键值数据库Redis，从高性能键值数据库Redis中查询使用预设的各素材区域图像。原始图像的一个区域为眉毛，相应区域的预设的卡通素材也为眉毛。
可选地，面部器官区域的每个关键点与素材区域的每个关键点之间可以得到卡方距离，这样就可以构成COST成本矩阵，成本矩阵的每个元素表示面部器官区域的某个关键点与素材区域的某个关键点之间的卡方距离。卡方距离用来衡量面部器官区域的某个关键点与素材区域的某个关键点之间的差异性。
可选地，根据原始图像的每一面部器官区域的形状上下文直方图特征、原始图像的头发区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，通过卡方距离计算，得到针对原始图像的每一面部器官区域的多个成本矩阵和针对原始图像的头发区域的多个成本矩阵，预设的各素材区域图像包括各面部器官素材图像和头发素材图像，成本矩阵的每个元素表征区域的一个关键点与相应区域的一个素材区域图像的一个关键点的卡方距离。
S104，根据针对各个区域的多个成本矩阵，从各素材区域图像中筛选出与原始图像每一区域相匹配的素材区域图像。
可选地，根据原始图像的任一区域对应的N个成本矩阵，为任一区域的每个关键点都匹配到预设的任一素材区域图像中的与每个关键点的卡方距离最小的关键点，将任一区域的所有关键点对应的最小卡方距离相加作为任一区域与任一素材区域图像之间的成本值，得到任一区域与预设的N个素材区域图像之间的N个成本值，N为正整数，将N个成本值中最小的成本值对应的素材区域图像作为与任一区域相匹配的素材区域图像。
可选地，根据原始图像的任一面部器官区域对应的N个成本矩阵，通过Hungarian匈牙利算法得到任一面部器官与预设的N个面部器官素材区域图像之间的N个第一成本值，N为正整数，将N个第一成本值中最小的第一成本值对应的面部器官素材区域图像作为与任一面部器官相匹配的素材区域图像。
可选地，根据原始图像的头发区域对应的M个成本矩阵，通过Hungarian匈牙利算法得到原始图像的头发区域与预设的M个头发素材区域图像之间的M个第二成本值，M为正整数，将M个第二成本值中最小的第二成本值对应的头发素材区域图像作为与原始图像的头发相匹配的素材区域图像。
S105，将与原始图像每一区域相匹配的素材区域图像进行拼接，得到原始图像对应的目标图像。
可选地，将与原始图像每一区域相匹配的素材区域图像按照对应区域在原图像中的位置进行拼接，得到原始图像对应的目标图像。
可选地，确定预设的各素材区域图像的形状上下文直方图特征的方式，包括：
获取各素材区域图像；
通过alpha阿尔法通道提取各素材区域图像的轮廓；
对各素材区域图像的轮廓进行采样，确定各素材区域图像的多个关键点；
将各素材区域图像的多个关键点输入预设的形状上下文向量提取器进行形状上下文直方图特征提取，得到预设的各素材区域图像的形状上下文直方图特征。
可选地，alpha阿尔法通道是一个8位的灰度通道，该通道用256级灰度来记录图像中的透明度信息，定义透明、不透明和半透明区域，其中白表示不透明，黑表示透明，灰表示半透明。
可选地，素材为卡通素材，将与原始图像每一区域相匹配的素材区域图像进行拼接，得到原始图像对应的目标图像，包括：
将与原始图像每一区域相匹配的卡通素材区域图像进行拼接，得到原始图像对应的卡通图像。
可选地，针对人身体的每个器官，例如眉毛、鼻子、嘴巴等，以及头发，都设计好很多卡通素材。针对输入的一张真人的图像，从素材库中寻找最相似的卡通素材，将这些最相似的卡通素材拼接到一起，产出一张卡通化的人物图像，该卡通化的人物图像与真人的图像有较大的相似度。
本申请实施例中，获取原始图像各区域的多个关键点，各区域对应人像中不同的各个身体部位；根据各区域的多个关键点，确定原始图像各区域的形状上下文直方图特征；根据原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确定针对各个区域的多个成本矩阵；根据针对各个区域的多个成本矩阵，从各素材区域图像中筛选出与所述原始图像每一区域相匹配的素材区域图像；将与原始图像每一区域相匹配的素材区域图像进行拼接，得到原始图像对应的目标图像。如此，实现了快速的为原始图像各区域匹配到对应的素材区域图像，将匹配到的素材区域图像进行拼接，从而得到的目标图像与原始图像在整体和各区域上都很相似，产生了美化的效果，显著地提升了用户体验。
本申请实施例中提供了另一种图像处理的方法，该方法的流程示意图如图2所示，该方法包括：
S201，获取真人图像的面部器官的稠密关键点。
可选地，原始图像为真人图像。稠密关键点是普通关键点的升级版，一个人的面部器官有81个或106个普通关键点，一个人的面部器官有1000个稠密关键点，面部器官 都有数量不等的稠密关键点，例如，面部器官中的脸的轮廓上有273个稠密关键点，面部器官中的眉毛区域的轮廓上有64个稠密关键点，面部器官中的眼睛区域的轮廓上有63个稠密关键点。
S202，将真人图像的面部器官的稠密关键点进行分类，得到真人图像的每一面部器官区域的多个稠密关键点。
可选地，对一个人的面部器官的稠密关键点进行分类，得到面部器官中的每一面部器官区域的稠密关键点，例如左眉区域的轮廓上的稠密关键点，并对左眉区域的轮廓上的稠密关键点进行有效采样到一个预设的数量，该数量为第一数值。
可选地，将卡通素材区域图像的形状上下文直方图特征存入到高性能键值数据库Redis中作查询使用。Redis支持的键值类型有：String字符类型、map散列类型、list列表类型、set集合类型和sortedset有序集合类型。
可选地，眉毛区域的轮廓上有64个稠密关键点，但是高性能键值数据库Redis中的眉毛区域的轮廓上只有30个普通关键点，因此，需要对眉毛区域的轮廓上的64个稠密关键点进行采样，得到30个稠密关键点，这样眉毛区域的轮廓上的30个稠密关键点的数量与高性能键值数据库Redis中的眉毛区域的轮廓上的30个普通关键点的数量相同，第一数值取值为30。
S203，获取真人图像的头发区域的灰度图。
S204，根据真人图像的头发区域的灰度图，确定真人图像的头发区域的轮廓，对真人图像的头发区域的轮廓进行采样，得到真人图像的头发区域的多个区域关键点。
S205，根据真人图像的每一面部器官区域的多个稠密关键点和真人图像的头发区域的多个区域关键点，得到真人图像的每一面部器官区域的形状上下文直方图特征和真人图像的头发区域的形状上下文直方图特征。
可选地，将真人图像的每一面部器官区域的多个稠密关键点和真人图像的头发区域的多个区域关键点，输入预设的形状上下文向量提取器进行形状上下文直方图特征提取，得到真人图像的每一面部器官区域的形状上下文直方图特征和真人图像的头发区域的形状上下文直方图特征。
S206，从Redis中调用卡通素材区域图像的形状上下文直方图特征，通过卡方距离计算，得到针对真人图像的每一面部器官区域的多个成本矩阵和针对真人图像的头发区 域的多个成本矩阵。
可选地，对真人图像的每个部位，例如，面部器官区域、头发区域，从Redis中调用卡通素材区域图像的形状上下文直方图特征，根据真人图像的每一面部器官区域的形状上下文直方图特征、真人图像的头发区域的形状上下文直方图特征和相应区域的各卡通素材区域图像的形状上下文直方图特征，并使用卡方距离公式(1)计算相应的COST成本矩阵，公式(1)如下所示：
Figure PCTCN2020091046-appb-000001
其中，公式(1)中的x为卡通素材区域图像的形状上下文直方图特征中的向量表示的关键点，y为真人图像的每一面部器官区域的形状上下文直方图特征或真人图像的头发区域的形状上下文直方图特征中向量表示的关键点。面部器官区域的每个关键点与卡通素材区域的每个关键点之间通过公式(1)得到卡方距离，这样就可以构成COST成本矩阵，成本矩阵的每个元素表示面部器官区域的某个关键点与卡通素材区域的某个关键点之间的卡方距离。卡方距离用来衡量面部器官区域的某个关键点与卡通素材区域的某个关键点之间的差异性。
S207，根据针对真人图像各个区域的多个成本矩阵，从各卡通素材区域图像中筛选出与真人图像每一区域相匹配的卡通素材区域图像。
可选地，根据真人图像的任一面部器官区域对应的N个成本矩阵，通过Hungarian匈牙利算法得到任一面部器官与预设的N个面部器官卡通区域素材图像之间的N个第一成本值，例如，N取值为10，将10个第一成本值中最小的第一成本值对应的面部器官卡通素材区域图像作为与任一面部器官相匹配的卡通素材区域图像。不同面部器官可以对应有不同预设数量的素材，得到的第一成本值的个数也不同。为面部器官的每个关键点都匹配到面部器官素材图像与其卡方距离最小的关键点，将所有关键点的最小卡方距离相加作为面部器官与面部器官素材的第一成本值。
可选地，根据真人图像的头发区域对应的M个成本矩阵，通过Hungarian匈牙利算法得到真人图像的头发区域与预设的M个头发素材区域图像之间的M个第二成本值，例如，M取值为10，将10个第二成本值中最小的第二成本值对应的头发卡通素材区域图像作为与真人图像的头发相匹配的卡通素材区域图像。
S208，将与真人图像每一区域相匹配的卡通素材区域图像进行拼接，得到真人图像 对应的卡通图像。
应用本申请实施例，至少具有如下有益效果：
实现了快速的为真人的原始图像各区域匹配到对应的卡通素材区域图像，将匹配到的卡通素材区域图像进行拼接，从而得到真人的原始图像对应的卡通图像，卡通图像与真人的原始图像在整体和各区域上都很相似，产生了美化的效果，显著地提升了用户体验。
实施例二
基于相同的发明构思，本申请实施例还提供了一种图像处理的装置，该装置的结构示意图如图3所示，图像处理的装置30，包括第一处理模块301、第二处理模块302、第三处理模块303、第四处理模块304和第五处理模块305。
第一处理模块301，用于获取原始图像各区域的多个关键点，各区域对应人像中不同的各个身体部位；
第二处理模块302，用于根据各区域的多个关键点，确定原始图像各区域的形状上下文直方图特征；
第三处理模块303，用于根据原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确定针对各个区域的多个成本矩阵；
第四处理模块304，用于根据针对各个区域的多个成本矩阵，从各素材区域图像中筛选出与原始图像每一区域相匹配的素材区域图像；
第五处理模块305，用于将与原始图像每一区域相匹配的素材区域图像进行拼接，得到原始图像对应的目标图像。
可选地，当原始图像各区域为原始图像的各面部器官区域时，第一处理模块301，具体用于获取原始图像的多个稠密关键点；将原始图像的多个稠密关键点进行分类，得到原始图像的每一面部器官区域的多个稠密关键点；对原始图像的每一面部器官区域的多个稠密关键点进行采样，得到采样后的原始图像的每一面部器官区域的多个关键点。
可选地，当原始图像的区域为原始图像的头发区域时，第一处理模块301，具体用 于获取原始图像的头发区域的灰度图；根据原始图像的头发区域的灰度图，确定原始图像的头发区域的轮廓；对原始图像的头发区域的轮廓进行采样，得到原始图像的头发区域的多个区域关键点。
可选地，第二处理模块302，具体用于将原始图像的每一面部器官区域的多个关键点和原始图像的头发区域的多个区域关键点，输入预设的形状上下文向量提取器进行形状上下文直方图特征提取，得到原始图像的每一面部器官区域的形状上下文直方图特征和原始图像的头发区域的形状上下文直方图特征。
可选地，第三处理模块303，具体用于根据原始图像的每一面部器官区域的形状上下文直方图特征、原始图像的头发区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，通过卡方距离计算，得到针对原始图像的每一面部器官区域的多个成本矩阵和针对原始图像的头发区域的多个成本矩阵，预设的各素材区域图像包括各面部器官素材图像和头发素材图像，成本矩阵的每个元素表征区域的一个关键点与相应区域的一个素材区域图像的一个关键点的卡方距离。
可选地，第四处理模块304，具体用于根据原始图像的任一区域对应的N个成本矩阵，为任一区域的每个关键点都匹配到预设的任一素材区域图像中的与每个关键点的卡方距离最小的关键点，将任一区域的所有关键点对应的最小卡方距离相加作为任一区域与任一素材区域图像之间的成本值，得到任一区域与预设的N个素材区域图像之间的N个成本值，N为正整数，将N个成本值中最小的成本值对应的素材区域图像作为与任一区域相匹配的素材区域图像。
可选地，确定预设的各素材区域图像的形状上下文直方图特征的方式，包括：
获取各素材区域图像；
通过阿尔法通道提取各素材区域图像的轮廓；
对各素材区域图像的轮廓进行采样，确定各素材区域图像的多个关键点；
将各素材区域图像的多个关键点输入预设的形状上下文向量提取器进行形状上下文直方图特征提取，得到预设的各素材区域图像的形状上下文直方图特征。
可选地，素材为卡通素材，第五处理模块305，具体用于将与原始图像每一区域相匹配的卡通素材区域图像进行拼接，得到原始图像对应的卡通图像。
应用本申请实施例，至少具有如下有益效果：
获取原始图像各区域的多个关键点，各区域对应人像中不同的各个身体部位；根据各区域的多个关键点，确定原始图像各区域的形状上下文直方图特征；根据原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确定针对各个区域的多个成本矩阵；根据针对各个区域的多个成本矩阵，从各素材区域图像中筛选出与所述原始图像每一区域相匹配的素材区域图像；将与原始图像每一区域相匹配的素材区域图像进行拼接，得到原始图像对应的目标图像。如此，实现了快速的为原始图像各区域匹配到对应的素材区域图像，将匹配到的素材区域图像进行拼接，从而得到的目标图像与原始图像在整体和各区域上都很相似，产生了美化的效果，显著地提升了用户体验。
本申请实施例提供的图像处理的装置中未详述的内容，可参照上述实施例一提供的图像处理的方法，本申请实施例提供的图像处理的装置能够达到的有益效果与上述实施例一提供的图像处理的方法相同，在此不再赘述。
实施例三
基于相同的发明构思，本申请实施例还提供了一种电子设备，该电子设备的结构示意图如图4所示，该电子设备7000包括至少一个处理器7001、存储器7002和总线7003，至少一个处理器7001均与存储7002电连接；存储器7002被配置用于存储有至少一个计算机可执行指令，处理器7001被配置用于执行该至少一个计算机可执行指令，从而执行如本申请实施例一中任意一个实施例或任意一种可选实施方式提供的任意一种图像处理的方法的步骤。
进一步，处理器7001可以是FPGA(Field－Programmable Gate Array，现场可编程门阵列)或者其它具有逻辑处理能力的器件，如MCU(Microcontroller Unit，微控制单元)、CPU(Central Process Unit，中央处理器)。
应用本申请实施例，至少具有如下有益效果：
获取原始图像各区域的多个关键点，各区域对应人像中不同的各个身体部位；根据各区域的多个关键点，确定原始图像各区域的形状上下文直方图特征；根据原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确定针对各个区域的多个成本矩阵；根据针对各个区域的多个成本矩阵，从各素材区域图像中筛选出与所述原始图像每一区域相匹配的素材区域图像；将与原始图像每 一区域相匹配的素材区域图像进行拼接，得到原始图像对应的目标图像。如此，实现了快速的为原始图像各区域匹配到对应的素材区域图像，将匹配到的素材区域图像进行拼接，从而得到的目标图像与原始图像在整体和各区域上都很相似，产生了美化的效果，显著地提升了用户体验。
实施例四
基于相同的发明构思，本申请实施例还提供了一种计算机可读存储介质，例如图4中的存储器7002，其中存储有计算机程序7002a，该计算机程序用于被处理器执行时实现本申请实施例一中任意一个实施例或任意一种图像处理的方法的步骤。
本申请实施例提供的计算机可读存储介质包括但不限于任何类型的盘(包括软盘、硬盘、光盘、CD-ROM、和磁光盘)、ROM(Read-Only Memory，只读存储器)、RAM(Random Access Memory，随即存储器)、EPROM(Erasable Programmable Read-Only Memory，可擦写可编程只读存储器)、EEPROM(Electrically Erasable Programmable Read-Only Memory，电可擦可编程只读存储器)、闪存、磁性卡片或光线卡片。也就是，可读存储介质包括由设备(例如，计算机)以能够读的形式存储或传输信息的任何介质。
应用本申请实施例，至少具有如下有益效果：
获取原始图像各区域的多个关键点，各区域对应人像中不同的各个身体部位；根据各区域的多个关键点，确定原始图像各区域的形状上下文直方图特征；根据原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确定针对各个区域的多个成本矩阵；根据针对各个区域的多个成本矩阵，从各素材区域图像中筛选出与所述原始图像每一区域相匹配的素材区域图像；将与原始图像每一区域相匹配的素材区域图像进行拼接，得到原始图像对应的目标图像。如此，实现了快速的为原始图像各区域匹配到对应的素材区域图像，将匹配到的素材区域图像进行拼接，从而得到的目标图像与原始图像在整体和各区域上都很相似，产生了美化的效果，显著地提升了用户体验。
本技术领域技术人员可以理解，可以用计算机程序指令来实现这些结构图和/或框图和/或流图中的每个框以及这些结构图和/或框图和/或流图中的框的组合。本技术领域技术人员可以理解，可以将这些计算机程序指令提供给通用计算机、专业计算机或其他 可编程数据处理方法的处理器来实现，从而通过计算机或其他可编程数据处理方法的处理器来执行本申请公开的结构图和/或框图和/或流图的框或多个框中指定的方案。
本技术领域技术人员可以理解，本申请中已经讨论过的各种操作、方法、流程中的步骤、措施、方案可以被交替、更改、组合或删除。进一步地，具有本申请中已经讨论过的各种操作、方法、流程中的其他步骤、措施、方案也可以被交替、更改、重排、分解、组合或删除。进一步地，现有技术中的具有与本申请中公开的各种操作、方法、流程中的步骤、措施、方案也可以被交替、更改、重排、分解、组合或删除。
以上所述仅是本申请的部分实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本申请原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本申请的保护范围。

Claims (12)

  1. 一种图像处理的方法，其特征在于，包括：
    获取原始图像各区域的多个关键点，所述各区域对应人像中不同的各个身体部位；
    根据所述各区域的多个关键点，确定所述原始图像各区域的形状上下文直方图特征；
    根据所述原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确定针对各个区域的多个成本矩阵；
    根据所述针对各个区域的多个成本矩阵，从所述各素材区域图像中筛选出与所述原始图像每一区域相匹配的素材区域图像；
    将与所述原始图像每一区域相匹配的素材区域图像进行拼接，得到所述原始图像对应的目标图像。
  2. 根据权利要求1所述的方法，其特征在于，当所述原始图像各区域为所述原始图像的各面部器官区域时，所述获取原始图像各区域的多个关键点，包括：
    获取所述原始图像的多个稠密关键点；
    将所述原始图像的多个稠密关键点进行分类，得到所述原始图像的每一面部器官区域的多个稠密关键点；
    对所述原始图像的每一面部器官区域的多个稠密关键点进行采样，得到采样后的原始图像的每一面部器官区域的多个关键点。
  3. 根据权利要求1所述的方法，其特征在于，当所述原始图像的区域为所述原始图像的头发区域时，所述获取原始图像各区域的多个关键点，包括：
    获取所述原始图像的头发区域的灰度图；
    根据所述原始图像的头发区域的灰度图，确定所述原始图像的头发区域的轮廓；
    对所述原始图像的头发区域的轮廓进行采样，得到所述原始图像的头发区域的多个区域关键点。
  4. 根据权利要求2所述的方法，其特征在于，所述根据所述各区域的多个关键点，确定所述原始图像各区域的形状上下文直方图特征，包括：
    将所述原始图像的每一面部器官区域的多个关键点和所述原始图像的头发区域的多个区域关键点，输入预设的形状上下文向量提取器进行形状上下文直方图特征提取，得到所述原始图像的每一面部器官区域的形状上下文直方图特征和所述原始图像的头发区域的形状上下文直方图特征。
  5. 根据权利要求4所述的方法，其特征在于，所述根据所述原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确 定针对各个区域的多个成本矩阵，包括：
    根据所述原始图像的每一面部器官区域的形状上下文直方图特征、所述原始图像的头发区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，通过卡方距离计算，得到针对所述原始图像的每一面部器官区域的多个成本矩阵和针对所述原始图像的头发区域的多个成本矩阵，所述预设的各素材区域图像包括各面部器官素材图像和头发素材图像，所述成本矩阵的每个元素表征所述区域的一个关键点与相应区域的一个素材区域图像的一个关键点的卡方距离。
  6. 根据权利要求5所述的方法，其特征在于，根据针对任一区域的多个成本矩阵，从所述各素材区域图像中筛选出与所述任一区域相匹配的素材区域图像，包括：
    根据所述原始图像的任一区域对应的N个成本矩阵，为所述任一区域的每个关键点都匹配到预设的任一素材区域图像中的与所述每个关键点的卡方距离最小的关键点，将所述任一区域的所有关键点对应的最小卡方距离相加作为所述任一区域与所述任一素材区域图像之间的成本值，得到所述任一区域与预设的N个素材区域图像之间的N个成本值，N为正整数，将N个成本值中最小的成本值对应的素材区域图像作为与所述任一区域相匹配的素材区域图像。
  7. 根据权利要求1所述的方法，其特征在于，确定所述预设的各素材区域图像的形状上下文直方图特征的方式，包括：
    获取各素材区域图像；
    通过阿尔法通道提取所述各素材区域图像的轮廓；
    对所述各素材区域图像的轮廓进行采样，确定所述各素材区域图像的多个关键点；
    将所述各素材区域图像的多个关键点输入预设的形状上下文向量提取器进行形状上下文直方图特征提取，得到所述预设的各素材区域图像的形状上下文直方图特征。
  8. 根据权利要求1所述的方法，其特征在于，所述素材为卡通素材，所述将与所述原始图像每一区域相匹配的素材区域图像进行拼接，得到所述原始图像对应的目标图像，包括：
    将与所述原始图像每一区域相匹配的卡通素材区域图像进行拼接，得到所述原始图像对应的卡通图像。
  9. 一种图像处理的装置，其特征在于，包括：
    第一处理模块，用于获取原始图像各区域的多个关键点，所述各区域对应人像中不同的各个身体部位；
    第二处理模块，用于根据所述各区域的多个关键点，确定所述原始图像各区域的形状上下文直方图特征；
    第三处理模块，用于根据所述原始图像各区域的形状上下文直方图特征和相应区域的预设的各素材区域图像的形状上下文直方图特征，确定针对各个区域的多个成本矩阵；
    第四处理模块，用于根据所述针对各个区域的多个成本矩阵，从所述各素材区域图像中筛选出与所述原始图像每一区域相匹配的素材区域图像；
    第五处理模块，用于将与所述原始图像每一区域相匹配的素材区域图像进行拼接，得到所述原始图像对应的目标图像。
  10. 一种电子设备，其特征在于，包括：处理器、存储器；
    所述存储器，用于存储计算机程序；
    所述处理器，用于通过调用所述计算机程序，执行上述权利要求1-8中任一项所述的图像处理的方法。
  11. 一种计算机可读存储介质，其特征在于，存储有计算机程序，所述计算机程序用于被处理器执行时实现如权利要求1-8中任一项所述的图像处理的方法。
  12. 一种计算机程序，包括计算机可读代码，当所述计算机可读代码在计算处理设备上运行时，导致所述计算处理设备执行根据权利要求1-8中任一项所述的图像处理的方法。
PCT/CN2020/091046 2019-06-27 2020-05-19 图像处理的方法、装置、设备及计算机可读存储介质 WO2020259129A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/612,750 US20220245777A1 (en) 2019-06-27 2020-05-19 Image processing method, apparatus and device and computer-readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910570341.5 2019-06-27
CN201910570341.5A CN110288516A (zh) 2019-06-27 2019-06-27 图像处理的方法、装置、设备及计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2020259129A1 true WO2020259129A1 (zh) 2020-12-30

Family

ID=68019366

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/091046 WO2020259129A1 (zh) 2019-06-27 2020-05-19 图像处理的方法、装置、设备及计算机可读存储介质

Country Status (3)

Country Link
US (1) US20220245777A1 (zh)
CN (1) CN110288516A (zh)
WO (1) WO2020259129A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110288516A (zh) * 2019-06-27 2019-09-27 北京迈格威科技有限公司 图像处理的方法、装置、设备及计算机可读存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104915634A (zh) * 2015-02-16 2015-09-16 百度在线网络技术（北京）有限公司 基于人脸识别技术的图像生成方法和装置
CN105096353A (zh) * 2014-05-05 2015-11-25 腾讯科技（深圳）有限公司 一种图像处理方法及装置
CN108717719A (zh) * 2018-05-23 2018-10-30 腾讯科技（深圳）有限公司 卡通人脸图像的生成方法、装置及计算机存储介质
CN109299720A (zh) * 2018-07-13 2019-02-01 沈阳理工大学 一种基于轮廓片段空间关系的目标识别方法
CN110288516A (zh) * 2019-06-27 2019-09-27 北京迈格威科技有限公司 图像处理的方法、装置、设备及计算机可读存储介质

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101567051B (zh) * 2009-06-03 2012-08-22 复旦大学 一种基于特征点的图像配准方法
CN101650784B (zh) * 2009-09-23 2012-10-24 南京大学 一种利用结构上下文特征进行图像匹配的方法
CN102622420B (zh) * 2012-02-22 2013-10-30 哈尔滨工程大学 基于颜色特征和形状上下文的商标图像检索方法
CN103049340A (zh) * 2012-10-26 2013-04-17 中山大学 基于纹理上下文约束的视觉词汇的图像超分辨率重建方法
CN103854278A (zh) * 2012-12-06 2014-06-11 五邑大学 基于连通区域质心形状上下文的印刷电路板图像配准方法
CN103679200A (zh) * 2013-12-14 2014-03-26 复旦大学 一种用于图像匹配、识别、检索的频域形状描述方法
US20150221048A1 (en) * 2014-02-06 2015-08-06 Frederic Zenhausern Method and system for matching a gear and travel storage for processing a transaction in an on-demand service system
US20180095174A1 (en) * 2016-09-30 2018-04-05 Faro Technologies, Inc. Three-dimensional coordinate measuring device
CN107103323B (zh) * 2017-03-09 2020-06-16 广东顺德中山大学卡内基梅隆大学国际联合研究院 一种基于图像轮廓特征的目标识别方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105096353A (zh) * 2014-05-05 2015-11-25 腾讯科技（深圳）有限公司 一种图像处理方法及装置
CN104915634A (zh) * 2015-02-16 2015-09-16 百度在线网络技术（北京）有限公司 基于人脸识别技术的图像生成方法和装置
CN108717719A (zh) * 2018-05-23 2018-10-30 腾讯科技（深圳）有限公司 卡通人脸图像的生成方法、装置及计算机存储介质
CN109299720A (zh) * 2018-07-13 2019-02-01 沈阳理工大学 一种基于轮廓片段空间关系的目标识别方法
CN110288516A (zh) * 2019-06-27 2019-09-27 北京迈格威科技有限公司 图像处理的方法、装置、设备及计算机可读存储介质

Also Published As

Publication number Publication date
US20220245777A1 (en) 2022-08-04
CN110288516A (zh) 2019-09-27

Similar Documents

Publication Publication Date Title
Ranjan et al. A fast and accurate system for face detection, identification, and verification
Ban et al. Face detection based on skin color likelihood
Cheng et al. Global contrast based salient region detection
Xiao et al. Content-adaptive superpixel segmentation
CN106485196B (zh) 视觉搜索方法、对应的系统、设备和计算机程序产品
WO2019136894A1 (en) Methods and systems for face alignment
US20220222918A1 (en) Image retrieval method and apparatus, storage medium, and device
Sajid et al. Data augmentation‐assisted makeup‐invariant face recognition
Aung et al. Face detection in real time live video using yolo algorithm based on Vgg16 convolutional neural network
WO2015078183A1 (zh) 图像身份识别方法和相关装置及身份识别系统
WO2021169257A1 (zh) 人脸识别
US11574392B2 (en) Automatically merging people and objects from multiple digital images to generate a composite digital image
Shu et al. Age progression: Current technologies and applications
WO2021127916A1 (zh) 脸部情感识别方法、智能装置和计算机可读存储介质
CN106203448B (zh) 一种基于非线性尺度空间的场景分类方法
CN112115805A (zh) 带双模态难挖掘三元-中心损失的行人重识别方法及系统
Sharma et al. Voxel-based 3D occlusion-invariant face recognition using game theory and simulated annealing
CN111553838A (zh) 模型参数的更新方法、装置、设备及存储介质
WO2020259129A1 (zh) 图像处理的方法、装置、设备及计算机可读存储介质
CN111666976A (zh) 基于属性信息的特征融合方法、装置和存储介质
Ge et al. Deep and discriminative feature learning for fingerprint classification
Wu et al. A new bag-of-words model using multi-cue integration for image retrieval
Wang et al. A person reidentification algorithm based on improved siamese network and hard sample
Ahmed et al. Generating digital images of skin diseases based on deep learning
Ouannes et al. Comparative Study Based on De-Occlusion and Reconstruction of Face Images in Degraded Conditions.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20833273

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20833273

Country of ref document: EP

Kind code of ref document: A1