CN113129410A - Sketch image conversion method and related product - Google Patents
Sketch image conversion method and related product Download PDFInfo
- Publication number
- CN113129410A CN113129410A CN201911419866.5A CN201911419866A CN113129410A CN 113129410 A CN113129410 A CN 113129410A CN 201911419866 A CN201911419866 A CN 201911419866A CN 113129410 A CN113129410 A CN 113129410A
- Authority
- CN
- China
- Prior art keywords
- image
- sketch
- sketch image
- face
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 61
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000000903 blocking effect Effects 0.000 claims abstract description 10
- 230000015654 memory Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 10
- 238000013528 artificial neural network Methods 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 5
- 230000001815 facial effect Effects 0.000 claims 2
- 238000010586 diagram Methods 0.000 description 6
- 210000004709 eyebrow Anatomy 0.000 description 4
- 210000001747 pupil Anatomy 0.000 description 4
- 206010063385 Intellectualisation Diseases 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000002146 bilateral effect Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 210000000744 eyelid Anatomy 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Image Processing (AREA)
Abstract
The embodiment of the application provides a sketch image conversion method and a related product, wherein the method comprises the following steps: the method comprises the steps of carrying out blocking processing on a sketch image to obtain N sketch image blocks, wherein N is an integer greater than or equal to 2; acquiring the background brightness of each sketch image block in the N sketch image blocks, and determining a target matching threshold of each sketch image block according to the background brightness of each sketch image block; matching each sketch image block according to the target matching threshold of each sketch image block to obtain a target face image block of each sketch image block; and combining the target face image blocks of each sketch image block to obtain a face image of the sketch image. By the adoption of the method and the device, the conversion precision of the sketch image can be improved.
Description
Technical Field
The application relates to the technical field of image recognition, in particular to a sketch image conversion method and a related product.
Background
The automatic portrait synthesis technology has attracted attention in recent years, and can be applied to various fields such as judicial arts or digital codes. For example, in the judicial field, searching a photo database of the police for criminal suspects with sketch portraits is a very important application. Generally, a sketch portrait is spoken by a relevant person and then drawn by a professional painter, or synthesized by an image synthesis technique according to the description of the relevant person. Therefore, in order to truly restore the real scene of the person to be painted, the environment information (such as the environment brightness) where the person to be painted is located is usually added to the sketch image.
Therefore, when the person to be painted is in a dark environment, the drawn face sketch image is darker in brightness, the drawn sketch image is low in definition, and when the sketch image is converted into the face image, abundant face features are difficult to extract, so that the conversion accuracy of the sketch image is low.
Disclosure of Invention
The embodiment of the application provides a sketch image conversion method and a related product. And through the blocking processing, the target matching threshold of each sketch image block is used for converting the sketch image, so that the conversion precision is improved.
In a first aspect, an embodiment of the present application provides a sketch image conversion method, including:
the method comprises the steps of carrying out blocking processing on a sketch image to obtain N sketch image blocks, wherein N is an integer greater than or equal to 2;
acquiring the background brightness of each sketch image block in the N sketch image blocks, and determining a target matching threshold of each sketch image block according to the background brightness of each sketch image block;
matching each sketch image block according to the target matching threshold of each sketch image block to obtain a target face image block of each sketch image block;
and combining the target face image blocks of each sketch image block to obtain a face image of the sketch image.
In a second aspect, an embodiment of the present application provides a pixilated image conversion apparatus, including:
the block unit is used for carrying out block processing on the sketch image to obtain N sketch image blocks, wherein N is an integer greater than or equal to 2;
the determining unit is used for acquiring the background brightness of each sketch image block in the N sketch image blocks and determining a target matching threshold of each sketch image block according to the background brightness of each sketch image block;
the matching unit is used for matching each pixel image block according to the target matching threshold of each pixel image block to obtain a target face image block of each pixel image block;
and the combination unit is used for combining the target face image blocks of each sketch image block to obtain the face image of the sketch image.
In a third aspect, embodiments of the present application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for performing the steps in the method according to the first aspect.
In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, which stores a computer program, where the computer program makes a computer execute the method according to the first aspect.
In a fifth aspect, embodiments of the present application provide a computer program product comprising a non-transitory computer-readable storage medium storing a computer program, the computer being operable to cause a computer to perform the method according to the first aspect.
The embodiment of the application has the following beneficial effects:
it can be seen that, in the embodiment of the present application, the block processing is performed on the sketch image, the target matching threshold of each sketch image block is obtained according to the background brightness of each sketch image block, and the target matching threshold of each sketch image block is used to match each sketch image, so that the target face image block corresponding to each sketch image block can be accurately determined, and further, the conversion efficiency and the accuracy of the sketch image are improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic flowchart of a sketch image conversion according to an embodiment of the present application;
FIG. 2 is a schematic flow chart of another sketch image conversion provided in the embodiment of the present application;
FIG. 3 is a schematic flowchart of another sketch image conversion provided in an embodiment of the present application;
fig. 4 is a schematic structural diagram of a pixel image conversion apparatus according to an embodiment of the present disclosure;
fig. 5 is a block diagram illustrating functional units of a pixel image conversion apparatus according to an embodiment of the present disclosure.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms "first," "second," "third," and "fourth," etc. in the description and claims of this application and in the accompanying drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "include" and "have," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, result, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
The sketch image conversion device in the application may include a smart Phone (such as an Android Phone, an iOS Phone, a Windows Phone, etc.), a tablet computer, a palm computer, a notebook computer, a Mobile Internet device MID (Mobile Internet Devices, abbreviated as MID) or a wearable device, etc. The above-mentioned sketch image converting apparatus is merely an example, and is not exhaustive, and includes but is not limited to the above-mentioned sketch image converting apparatus. In practical application, the method can further comprise the following steps: intelligent vehicle-mounted terminal, computer equipment and the like.
Referring to fig. 1, fig. 1 is a sketch image conversion method according to an embodiment of the present disclosure. The method is applied to a sketch image conversion device. The method includes, but is not limited to, the steps of:
101: the sketch image conversion device carries out block processing on the sketch image to obtain N sketch image blocks.
Wherein N is an integer greater than or equal to 2.
The sketch image is a sketch image containing a human face. Specifically, the sketch image may be partitioned in a bilateral symmetry manner to obtain N/2 sketch image block pairs, where each sketch image block pair includes sketch image blocks that are bilateral symmetry to each other.
102: the sketch image conversion device obtains the background brightness of each sketch image block in the N sketch image blocks, and determines the target matching threshold of each sketch image block according to the background brightness of each sketch image block.
The background brightness is the brightness value of each sketch image block.
And determining a target matching threshold value of each pixel image block according to the mapping relation between the background brightness and the matching threshold value.
103: and the sketch image conversion device matches each sketch image block according to the target matching threshold of each sketch image block to obtain the target face image block of each sketch image block.
Specifically, feature extraction is carried out on each pixel image block to obtain a feature vector of each pixel image block; matching the feature vector of each sketch image block with each face sketch image block template to obtain a matching value corresponding to each face sketch image block template; and taking the human face sketch image block template with the matching value larger than the target matching value of the sketch image block as a target human face image block of the sketch image block.
The feature extraction of each sketch image block can be performed through a neural network, and the neural network can be one or a combination of a convolutional neural network, a cyclic neural network and a long-term and short-term memory network.
Optionally, each pixel image block may be matched by:
performing feature extraction on feature points of a sketch image block A to obtain a first feature vector, wherein the sketch image block A is any sketch image block in the N sketch image blocks;
extracting the features of the outline of the pixel image block A to obtain a second feature vector;
splicing the first characteristic vector and the second characteristic vector to obtain a target characteristic vector of the pixel image block A;
and matching the target feature vector of the sketch image block A with the target feature vector of each face template to obtain a matching value corresponding to each face template, wherein the target feature vector of each face template is obtained by splicing the feature vector corresponding to the feature point on each face template and the feature vector corresponding to the outline.
104: and the sketch image conversion device combines the target face image blocks of each sketch image block to obtain the face image of the sketch image.
And finally, combining the target face image blocks of each sketch image block to obtain a face image corresponding to the sketch image. If the number of the target face image blocks of a certain sketch image block is multiple, the target face image block with the largest matching value in the multiple target face image blocks can be used as a final target face image block; and combining the final target face image block of the sketch image block with target face image blocks corresponding to other sketch image blocks to obtain a face image corresponding to the sketch image.
It can be seen that, in the embodiment of the present application, the block processing is performed on the sketch image, the target matching threshold of each sketch image block is obtained according to the background brightness of each sketch image block, and the target matching threshold of each sketch image block is used to match each sketch image, so that the target face image block corresponding to each sketch image block can be accurately determined, and further, the conversion efficiency and the accuracy of the sketch image are improved.
In a possible embodiment, when there are a plurality of target face image blocks of a certain sketch image block, for example, when there are a plurality of target face image blocks of a sketch image block a (the sketch image block a is any one of the N sketch image blocks), the face image of the sketch image may also be obtained by:
combining a plurality of target face image blocks of a sketch image block A into N-1 target face image blocks corresponding to N-1 sketch image blocks respectively to obtain a plurality of face images, wherein the N-1 sketch image blocks are all sketch image blocks except the sketch image block A in the N sketch image blocks;
determining the coordination proportion of each face image in a plurality of face images;
and taking the face image with the maximum coordination ratio in the face images as the face image of the sketch image.
It should be noted that, when there are a plurality of target face image blocks corresponding to a plurality of sketch blocks in the N sketch blocks, the target face image blocks corresponding to each sketch block need to be combined to obtain a plurality of face images, then, the coordination ratio of each face image is calculated, and the face image of the sketch image is selected according to the coordination ratio.
Optionally, a neural network may be used to perform feature extraction on each face image in the plurality of face images to obtain a feature vector of each face image; classifying each face image according to the feature vector of each face image and each face image library, namely comparing each face image with each face template in the face image library one by one to obtain a matching value corresponding to each face template, and then taking the maximum matching value as a target matching value of the face image; and the target matching value of each face image is used as the coordination proportion of each face image. Namely, each face image is matched with the template, so that the face image with the real identity can be matched, and the higher the coordination proportion of the face image is.
Optionally, a distance between any two preset feature points on a face image block a on each face image may be obtained to obtain a plurality of first distances, where the face image block a is a target face image block corresponding to the sketch image block a, where the preset feature points correspond to a face region to which the face image block a belongs, generally, 68 preset feature points (e.g., a left pupil, a right pupil, a left eyebrow, and a right eyebrow) may be divided on the face image, and then all the preset feature points on the face image block a are determined according to the face region to which the face image block a belongs, and then, a distance between any two preset feature points in all the preset feature points on the face image block a is obtained to obtain a plurality of first distances; acquiring the distance between any two preset feature points on a face image block B on each face image to obtain a plurality of second distances, wherein the face image block B is a target face image block corresponding to a sketch image block B, and the sketch image block B is an image block which is symmetrical to the sketch image block A in the N sketch image blocks; and acquiring a difference value between the corresponding first distance and second distance to obtain a plurality of difference values, namely acquiring a difference value between the first distance and the second distance corresponding to the preset feature points (the preset feature points are symmetrical feature points). For example, if the first distance is the distance from the left pupil to the left eyebrow center, the corresponding second distance is the distance from the right pupil to the right eyebrow center; and taking the standard deviation of the plurality of difference values as the coordination proportion of each face image. Of course, the variance of a plurality of differences may be used as the coordination ratio of each face image.
In a possible implementation, before the block processing is performed on the sketch image to obtain N sketch image blocks, the method further includes:
acquiring voice data, performing semantic recognition on the voice data to obtain semantic information of the voice data (the semantic information can be extracted by using a GRU network), extracting keywords from the semantic information to obtain a plurality of keywords, and determining a sketch descriptor corresponding to each keyword in the plurality of keywords according to a mapping relation between the keywords and the sketch descriptor; and obtaining the sketch image according to the plurality of sketch descriptors, namely combining the plurality of sketch descriptors to obtain the sketch image.
Among them, sketch descriptors include but are not limited to: left eye, right eye, left ear, right ear, double eyelid, single eyelid, eye, nose, mouth, left eyebrow, right eyebrow, ear.
In the embodiment, the sketch image can be automatically generated through the voice data, the generation rate of the sketch image is improved, and the intellectualization of converting the sketch image into the face image is further realized.
In a possible implementation, before the block processing is performed on the sketch image to obtain N sketch image blocks, the method further includes:
acquiring a Hessian matrix of each pixel point on the sketch image to obtain a first component image, a second component image and a third component image corresponding to the sketch image; specifically, high-frequency directional decomposition is performed on each pixel point in the sketch image to obtain a first component of each pixel point in the horizontal direction, a second component of each pixel point in the vertical direction and a third component of each pixel point in the diagonal direction, wherein the high-frequency directional decomposition is performed on each pixel point to substantially solve a Hessian (Hessian) matrix of each pixel point, that is, a second-order partial differential of each pixel point in the horizontal direction, a second-order partial differential of each pixel point in the vertical direction and a mixed partial differential of each pixel point in the diagonal direction are respectively solved, the second-order partial differential of each pixel point in the horizontal direction is used as the first component of each pixel point, the second-order partial differential of each pixel point in the vertical direction is used as the second component, and the mixed partial differential of each pixel point in the diagonal direction is used as the third component. Then, correspondingly forming first components of all pixel points in the sketch image in the horizontal direction into a first component image, correspondingly forming second components of all pixel points in the vertical direction into a second component image, and correspondingly forming third components of all pixel points in the diagonal direction into a third component image;
and superposing the sketch image with the first component image, the second component image and the third component image to obtain an enhanced sketch image.
Further, after the sketch image is enhanced, the enhanced sketch image may be divided to obtain N sketch image blocks.
In the embodiment of the application, the sketch image is enhanced, so that the enhanced image is clearer, the human face features are more obvious, and the conversion precision of the sketch image is improved.
Referring to fig. 2, fig. 2 is a schematic diagram illustrating another exemplary sketch image conversion method according to an embodiment of the present disclosure. The method is applied to a sketch image conversion device. The method includes, but is not limited to, the steps of:
201: the sketch image conversion device acquires voice data.
202: and the sketch image conversion device carries out semantic recognition on the voice data to obtain semantic information of the voice data.
203: and the sketch image conversion device extracts keywords from the semantic information to obtain a plurality of keywords.
204: the sketch image conversion device determines a sketch descriptor corresponding to each keyword in the plurality of keywords according to the mapping relation between the keywords and the sketch descriptors to obtain a plurality of sketch descriptors.
205: and the sketch image conversion device obtains a sketch image according to the sketch descriptors.
206: the sketch image conversion device carries out block processing on the sketch image to obtain N sketch image blocks.
207: the sketch image conversion device obtains the background brightness of each sketch image block in the N sketch image blocks, and determines the target matching threshold of each sketch image block according to the background brightness of each sketch image block.
208: and the sketch image conversion device matches each sketch image block according to the target matching threshold of each sketch image block to obtain the target face image block of each sketch image block.
209: and the sketch image conversion device combines the target face image blocks of each sketch image block to obtain the face image of the sketch image.
It should be noted that, the specific implementation of the steps of the method shown in fig. 2 can refer to the specific implementation of the method described in fig. 1, and will not be described here.
It can be seen that, in the embodiment of the present application, the block processing is performed on the sketch image, the target matching threshold of each sketch image block is obtained according to the background brightness of each sketch image block, and the target matching threshold of each sketch image block is used to match each sketch image, so that the target face image block corresponding to each sketch image block can be accurately determined, and the conversion efficiency and accuracy of the sketch image are improved; moreover, the sketch image can be automatically generated through voice data, the intellectualization of the conversion of the sketch image is realized, and the conversion efficiency of the sketch image is further improved.
Referring to fig. 3, fig. 3 is a schematic diagram illustrating another exemplary method applied to a sketch image conversion apparatus according to an embodiment of the present disclosure. The method includes, but is not limited to, the steps of:
301: the sketch image conversion device acquires voice data.
302: and the sketch image conversion device carries out semantic recognition on the voice data to obtain semantic information of the voice data.
303: and the sketch image conversion device extracts keywords from the semantic information to obtain a plurality of keywords.
304: the sketch image conversion device determines a sketch descriptor corresponding to each keyword in the plurality of keywords according to the mapping relation between the keywords and the sketch descriptors to obtain a plurality of sketch descriptors.
305: and the sketch image conversion device obtains a sketch image according to the sketch descriptors.
306: the sketch image conversion device obtains a Hessian matrix of each pixel point on the sketch image to obtain a first component image, a second component image and a third component image corresponding to the sketch image.
307: and the sketch image conversion device superposes the sketch image with the first component image, the second component image and the third component image to obtain an enhanced sketch image.
308: the sketch image conversion device carries out block processing on the sketch image to obtain N sketch image blocks.
309: the sketch image conversion device obtains the background brightness of each sketch image block in the N sketch image blocks, and determines the target matching threshold of each sketch image block according to the background brightness of each sketch image block.
310: and the sketch image conversion device matches each sketch image block according to the target matching threshold of each sketch image block to obtain the target face image block of each sketch image block.
311: and the sketch image conversion device combines the target face image blocks of each sketch image block to obtain the face image of the sketch image.
It should be noted that, for the specific implementation of the steps of the method shown in fig. 3, reference may be made to the specific implementation of the method described in fig. 1, and a description thereof is omitted here.
It can be seen that, in the embodiment of the present application, the block processing is performed on the sketch image, the target matching threshold of each sketch image block is obtained according to the background brightness of each sketch image block, and the target matching threshold of each sketch image block is used to match each sketch image, so that the target face image block corresponding to each sketch image block can be accurately determined, and the conversion efficiency and accuracy of the sketch image are improved; moreover, the sketch image can be automatically generated through voice data, the intellectualization of the conversion of the sketch image is realized, and the conversion efficiency of the sketch image is further improved; and before the block processing, the sketch image is enhanced so as to enhance the human face characteristics in the sketch image and further improve the conversion precision.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a pixel image conversion apparatus 400 according to an embodiment of the present disclosure, as shown in fig. 4, the pixel image conversion apparatus 400 includes a processor, a memory, a communication interface, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the processor, and the program includes instructions for performing the following steps:
the method comprises the steps of carrying out blocking processing on a sketch image to obtain N sketch image blocks, wherein N is an integer greater than or equal to 2;
acquiring the background brightness of each sketch image block in the N sketch image blocks, and determining a target matching threshold of each sketch image block according to the background brightness of each sketch image block;
matching each sketch image block according to the target matching threshold of each sketch image block to obtain a target face image block of each sketch image block;
and combining the target face image blocks of each sketch image block to obtain a face image of the sketch image.
When a plurality of target face image blocks of a sketch image block a are present, the sketch image block a is any one of the N sketch image blocks, and the face sketch image block of each sketch image block is combined to obtain a face image of the sketch image, where the program is specifically configured to execute instructions of the following steps:
combining the multiple target face blocks of the sketch image block A with N-1 target face image blocks corresponding to N-1 sketch image blocks respectively to obtain multiple face images, wherein the N-1 sketch image blocks are all sketch image blocks except the sketch image block A in the N sketch image blocks;
determining a coordination ratio of each face image in the plurality of face images;
and taking the face image with the maximum coordination proportion in the face images as the face image of the sketch image.
In one possible embodiment, the program is specifically configured to execute the following steps in determining the harmony scale of each of the plurality of face images:
extracting the features of each face image in the plurality of face images by using a neural network to obtain the feature vector of each face image;
classifying each face image according to the feature vector of each face image and a face image library to obtain a target matching value corresponding to each face image, and taking the target matching value of each face image as the coordination proportion of each face image.
In one possible embodiment, the program is specifically configured to execute the following steps in determining the harmony scale of each of the plurality of face images:
acquiring the distance between any two preset feature points on a face image block a on each face image to obtain a plurality of first distances, wherein the face image block a is a target face image block corresponding to the sketch image block A;
acquiring the distance between any two preset feature points on a face image block B on each face image to obtain a plurality of second distances, wherein the face image block B is a target face image block corresponding to a sketch image block B, and the sketch image block B is an image block which is symmetrical to the sketch image block A in the N sketch image blocks;
obtaining a difference value between the corresponding first distance and the second distance to obtain a plurality of difference values;
and taking the standard deviation of the plurality of difference values as the coordination proportion of each face image.
In a possible embodiment, before the block processing is performed on the sketch image to obtain N sketch image blocks, the program is further configured to execute the following steps:
acquiring voice data;
performing semantic recognition on the voice data to obtain semantic information of the voice data;
extracting keywords from the semantic information to obtain a plurality of keywords;
determining a sketch descriptor corresponding to each keyword in the plurality of keywords according to a mapping relation between the keywords and the sketch descriptors to obtain a plurality of sketch descriptors;
and obtaining the sketch image according to the plurality of sketch descriptors.
In a possible embodiment, before the block processing is performed on the sketch image to obtain N sketch image blocks, the program is further configured to execute the following steps:
acquiring a Hessian matrix of each pixel point on the sketch image to obtain a first component image, a second component image and a third component image corresponding to the sketch image;
superposing the sketch image with the first component image, the second component image and the third component image to obtain an enhanced sketch image;
in the aspect of performing block processing on a sketch image to obtain N sketch image blocks, the program is specifically configured to execute instructions for:
and carrying out blocking processing on the enhanced sketch image to obtain N sketch image blocks.
Referring to fig. 5, fig. 5 shows a block diagram of a possible functional unit of the pixel image converting apparatus 500 according to the above embodiment, and the pixel image converting apparatus 500 includes: a partitioning unit 510, a determining unit 520, a matching unit 530, a combining unit 540, wherein:
a blocking unit 510, configured to perform blocking processing on the sketch image to obtain N sketch image blocks, where N is an integer greater than or equal to 2;
a determining unit 520, configured to obtain a background brightness of each of the N sketch image blocks, and determine a target matching threshold of each sketch image block according to the background brightness of each sketch image block;
a matching unit 530, configured to match each of the pixel image blocks according to a target matching threshold of each of the pixel image blocks, to obtain a target face image block of each of the pixel image blocks;
and the combining unit 540 is configured to combine the target face image blocks of each sketch image block to obtain a face image of the sketch image.
In a possible implementation manner, when there are a plurality of target face image blocks of a sketch image block a, the sketch image block a is any one of the N sketch image blocks, and in terms of combining the face sketch image blocks of each sketch image block to obtain a face image of the sketch image, the combining unit 540 is specifically configured to:
combining the multiple target face blocks of the sketch image block A with N-1 target face image blocks corresponding to N-1 sketch image blocks respectively to obtain multiple face images, wherein the N-1 sketch image blocks are all sketch image blocks except the sketch image block A in the N sketch image blocks;
determining a coordination ratio of each face image in the plurality of face images;
and taking the face image with the maximum coordination proportion in the face images as the face image of the sketch image.
In a possible implementation manner, in determining the coordination ratio of each face image in the plurality of face images, the combining unit 540 is specifically configured to:
extracting the features of each face image in the plurality of face images by using a neural network to obtain the feature vector of each face image;
classifying each face image according to the feature vector of each face image and a face image library to obtain a target matching value corresponding to each face image, and taking the target matching value of each face image as the coordination proportion of each face image.
In a possible implementation manner, in determining the coordination ratio of each face image in the plurality of face images, the combining unit 540 is specifically configured to:
acquiring the distance between any two preset feature points on a face image block a on each face image to obtain a plurality of first distances, wherein the face image block a is a target face image block corresponding to the sketch image block A;
acquiring the distance between any two preset feature points on a face image block B on each face image to obtain a plurality of second distances, wherein the face image block B is a target face image block corresponding to a sketch image block B, and the sketch image block B is an image block which is symmetrical to the sketch image block A in the N sketch image blocks;
obtaining a difference value between the corresponding first distance and the second distance to obtain a plurality of difference values;
and taking the standard deviation of the plurality of difference values as the coordination proportion of each face image.
In a possible implementation, the sketch image converting device 500 further comprises an identifying unit 550, before the block processing is performed on the sketch image to obtain N sketch image blocks, the identifying unit 550 is configured to:
acquiring voice data;
performing semantic recognition on the voice data to obtain semantic information of the voice data;
extracting keywords from the semantic information to obtain a plurality of keywords;
determining a sketch descriptor corresponding to each keyword in the plurality of keywords according to a mapping relation between the keywords and the sketch descriptors to obtain a plurality of sketch descriptors;
and obtaining the sketch image according to the plurality of sketch descriptors.
In a possible implementation manner, before the block processing is performed on the sketch image, the sketch image converting apparatus 500 further includes an enhancing unit 560, and before obtaining N sketch image blocks, the enhancing unit 560 is configured to:
acquiring a Hessian matrix of each pixel point on the sketch image to obtain a first component image, a second component image and a third component image corresponding to the sketch image;
superposing the sketch image with the first component image, the second component image and the third component image to obtain an enhanced sketch image;
in terms of performing block processing on the sketch image to obtain N sketch image blocks, the block dividing unit 510 is specifically configured to:
and carrying out blocking processing on the enhanced sketch image to obtain N sketch image blocks.
Embodiments of the present application also provide a computer storage medium, which stores a computer program, where the computer program is executed by a processor to implement part or all of the steps of any one of the sketch image conversion methods as described in the above method embodiments.
Embodiments of the present application also provide a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any one of the sketch image conversion methods as described in the above method embodiments.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are exemplary embodiments and that the acts and modules referred to are not necessarily required in this application.
In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus may be implemented in other manners. For example, the above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of some interfaces, devices or units, and may be an electric or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may be implemented in the form of a software program module.
The integrated units, if implemented in the form of software program modules and sold or used as stand-alone products, may be stored in a computer readable memory. Based on such understanding, the technical solution of the present application may be substantially implemented or a part of or all or part of the technical solution contributing to the prior art may be embodied in the form of a software product stored in a memory, and including several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned memory comprises: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable memory, which may include: flash Memory disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
The foregoing detailed description of the embodiments of the present application has been presented to illustrate the principles and implementations of the present application, and the above description of the embodiments is only provided to help understand the method and the core concept of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application.
Claims (10)
1. A sketch image conversion method, comprising:
the method comprises the steps of carrying out blocking processing on a sketch image to obtain N sketch image blocks, wherein N is an integer greater than or equal to 2;
acquiring the background brightness of each sketch image block in the N sketch image blocks, and determining a target matching threshold of each sketch image block according to the background brightness of each sketch image block;
matching each sketch image block according to the target matching threshold of each sketch image block to obtain a target face image block of each sketch image block;
and combining the target face image blocks of each sketch image block to obtain a face image of the sketch image.
2. The method according to claim 1, wherein when there are a plurality of target face image blocks of a sketch image block a, the sketch image block a is any one of the N sketch image blocks, and the combining the face sketch image blocks of each sketch image block to obtain the face image of the sketch image comprises:
combining the multiple target face blocks of the sketch image block A with N-1 target face image blocks corresponding to N-1 sketch image blocks respectively to obtain multiple face images, wherein the N-1 sketch image blocks are all sketch image blocks except the sketch image block A in the N sketch image blocks;
determining a coordination ratio of each face image in the plurality of face images;
and taking the face image with the maximum coordination proportion in the face images as the face image of the sketch image.
3. The method of claim 2, wherein determining the harmony measure for each of the plurality of facial images comprises:
using a neural network to perform feature extraction on each face image in the plurality of face images to obtain a feature vector of each face image;
classifying each face image according to the feature vector of each face image and a face image library to obtain a target matching value corresponding to each face image, and taking the target matching value of each face image as the coordination proportion of each face image.
4. The method of claim 2, wherein determining the harmony measure for each of the plurality of facial images comprises:
acquiring the distance between any two preset feature points on a face image block a on each face image to obtain a plurality of first distances, wherein the face image block a is a target face image block corresponding to the sketch image block A on each face image;
acquiring the distance between any two preset feature points on a face image block B on each face image to obtain a plurality of second distances, wherein the face image block B is a target face image block corresponding to a sketch image block B on each face image, and the sketch image block B is a sketch image block symmetrical to the sketch image block A in the N sketch image blocks;
obtaining a difference value between the corresponding first distance and the second distance to obtain a plurality of difference values;
and taking the standard deviation of the plurality of difference values as the coordination proportion of each face image.
5. The method of any one of claims 1-4, wherein before the block processing the sketch image to obtain N sketch image blocks, the method further comprises:
acquiring voice data;
performing semantic recognition on the voice data to obtain semantic information of the voice data;
extracting keywords from the semantic information to obtain a plurality of keywords;
determining a sketch descriptor corresponding to each keyword in the plurality of keywords according to a mapping relation between the keywords and the sketch descriptors to obtain a plurality of sketch descriptors;
and obtaining the sketch image according to the plurality of sketch descriptors.
6. The method of any one of claims 1-5, wherein before the block processing the sketch image to obtain N sketch image blocks, the method further comprises:
acquiring a Hessian matrix of each pixel point on the sketch image to obtain a first component image, a second component image and a third component image corresponding to the sketch image;
superposing the sketch image with the first component image, the second component image and the third component image to obtain an enhanced sketch image;
the block processing is performed on the sketch image to obtain N sketch image blocks, and the block processing comprises the following steps:
and carrying out blocking processing on the enhanced sketch image to obtain N sketch image blocks.
7. A pixilated image conversion apparatus, comprising:
the block unit is used for carrying out block processing on the sketch image to obtain N sketch image blocks, wherein N is an integer greater than or equal to 2;
the determining unit is used for acquiring the background brightness of each sketch image block in the N sketch image blocks and determining a target matching threshold of each sketch image block according to the background brightness of each sketch image block;
the matching unit is used for matching each pixel image block according to the target matching threshold of each pixel image block to obtain a target face image block of each pixel image block;
and the combination unit is used for combining the target face image blocks of each sketch image block to obtain the face image of the sketch image.
8. The apparatus according to claim 7, wherein when there are a plurality of target face image blocks of a sketch image block a, the sketch image block a is any one of the N sketch image blocks, and in terms of combining the face sketch image blocks of each sketch image block to obtain the face image of the sketch image, the combining unit is specifically configured to:
combining the multiple target face blocks of the sketch image block A with N-1 target face image blocks corresponding to N-1 sketch image blocks respectively to obtain multiple face images, wherein the N-1 sketch image blocks are all sketch image blocks except the sketch image block A in the N sketch image blocks;
determining a coordination ratio of each face image in the plurality of face images;
and taking the face image with the maximum coordination proportion in the face images as the face image of the sketch image.
9. An electronic device comprising a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, the programs comprising instructions for performing the steps of the method of any of claims 1-6.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which is executed by a processor to implement the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911419866.5A CN113129410B (en) | 2019-12-31 | Sketch image conversion method and related product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911419866.5A CN113129410B (en) | 2019-12-31 | Sketch image conversion method and related product |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113129410A true CN113129410A (en) | 2021-07-16 |
| CN113129410B CN113129410B (en) | 2024-06-07 |
Family
ID=
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109785439A (en) * | 2018-12-27 | 2019-05-21 | 深圳云天励飞技术有限公司 | Human face sketch image generating method and Related product |
| CN109858355A (en) * | 2018-12-27 | 2019-06-07 | 深圳云天励飞技术有限公司 | Image processing method and Related product |
| CN110023989A (en) * | 2017-03-29 | 2019-07-16 | 华为技术有限公司 | A kind of generation method and device of sketch image |
| WO2019223102A1 (en) * | 2018-05-22 | 2019-11-28 | 平安科技（深圳）有限公司 | Method and apparatus for checking validity of identity, terminal device and medium |
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110023989A (en) * | 2017-03-29 | 2019-07-16 | 华为技术有限公司 | A kind of generation method and device of sketch image |
| WO2019223102A1 (en) * | 2018-05-22 | 2019-11-28 | 平安科技（深圳）有限公司 | Method and apparatus for checking validity of identity, terminal device and medium |
| CN109785439A (en) * | 2018-12-27 | 2019-05-21 | 深圳云天励飞技术有限公司 | Human face sketch image generating method and Related product |
| CN109858355A (en) * | 2018-12-27 | 2019-06-07 | 深圳云天励飞技术有限公司 | Image processing method and Related product |
Non-Patent Citations (1)
| Title |
|---|
| 曹林;李猛;张华;: "结合改进Surf特征的素描人脸识别", 计算机工程与应用, no. 21 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112052789B (en) | Face recognition method and device, electronic equipment and storage medium | |
| US20230081645A1 (en) | Detecting forged facial images using frequency domain information and local correlation | |
| CN111368685B (en) | Method and device for identifying key points, readable medium and electronic equipment | |
| CN107330408B (en) | Video processing method and device, electronic equipment and storage medium | |
| CN106682632B (en) | Method and device for processing face image | |
| US11257293B2 (en) | Augmented reality method and device fusing image-based target state data and sound-based target state data | |
| CN107333071A (en) | Video processing method and device, electronic equipment and storage medium | |
| EP3893125A1 (en) | Method and apparatus for searching video segment, device, medium and computer program product | |
| CN111275784B (en) | Method and device for generating image | |
| CN110188829B (en) | Neural network training method, target recognition method and related products | |
| CN108388878A (en) | The method and apparatus of face for identification | |
| CN111814655B (en) | Target re-identification method, network training method thereof and related device | |
| CN112487886A (en) | Method and device for identifying face with shielding, storage medium and terminal | |
| US20230143452A1 (en) | Method and apparatus for generating image, electronic device and storage medium | |
| WO2021249006A1 (en) | Method and apparatus for identifying authenticity of facial image, and medium and program product | |
| CN112818995B (en) | Image classification method, device, electronic equipment and storage medium | |
| CN113362263A (en) | Method, apparatus, medium, and program product for changing the image of a virtual idol | |
| CN112950640A (en) | Video portrait segmentation method and device, electronic equipment and storage medium | |
| CN110619334A (en) | Portrait segmentation method based on deep learning, architecture and related device | |
| CN109753873B (en) | Image processing method and related device | |
| CN113031813A (en) | Instruction information acquisition method and device, readable storage medium and electronic equipment | |
| CN111652878A (en) | Image detection method, image detection device, computer equipment and storage medium | |
| CN113129410B (en) | Sketch image conversion method and related product | |
| CN115861515A (en) | Three-dimensional face reconstruction method, computer program product and electronic device | |
| CN113129410A (en) | Sketch image conversion method and related product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |