CN114757816A - Sketch-based picture conversion method, device, equipment and storage medium - Google Patents
Sketch-based picture conversion method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN114757816A CN114757816A CN202210319444.6A CN202210319444A CN114757816A CN 114757816 A CN114757816 A CN 114757816A CN 202210319444 A CN202210319444 A CN 202210319444A CN 114757816 A CN114757816 A CN 114757816A
- Authority
- CN
- China
- Prior art keywords
- conversion
- picture
- sketch
- target
- sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000006243 chemical reaction Methods 0.000 title claims abstract description 382
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000000605 extraction Methods 0.000 claims abstract description 13
- 238000012549 training Methods 0.000 claims description 48
- 238000011156 evaluation Methods 0.000 claims description 17
- 238000013507 mapping Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 abstract description 3
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 241000282412 Homo Species 0.000 description 1
- 241000282414 Homo sapiens Species 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention belongs to the technical field of image processing, and discloses a sketch-based picture conversion method, device, equipment and storage medium. The method comprises the following steps: obtaining a sketch to be converted and a sketch conversion instruction of a user; determining a target picture type according to the sketch conversion instruction; performing feature extraction on the sketch to be converted to obtain target conversion features of the sketch to be converted; and performing picture conversion according to the target picture type and the target conversion characteristic to obtain a target conversion picture. By the mode, the accurate target conversion characteristic and the target picture type are determined through the sketch to be converted and the sketch conversion instruction of the user, and picture conversion is carried out based on the target picture type and the target conversion characteristic, so that the accuracy and the high efficiency of picture conversion are ensured, and the experience of the user is improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for converting pictures based on sketches.
Background
Sketching is one of the most natural and flexible ways for humans to express and convey information. With the increasing popularity of touch screen devices in recent years, the visual needs of human beings are expressed by drawing a few sketches on a touch screen, and the sketches become important man-machine interaction modes. Because sketches are highly abstract and sketches drawn by different people may vary greatly in accuracy and style of completion, the sketches-image translation problem still faces significant challenges.
The above is only for the purpose of assisting understanding of the technical aspects of the present invention, and does not represent an admission that the above is prior art.
Disclosure of Invention
The invention mainly aims to provide a sketch-based picture conversion method, a sketch-based picture conversion device, equipment and a storage medium, and aims to solve the technical problem that the prior art cannot convert an accurate picture according to a sketch-based picture.
In order to achieve the above object, the present invention provides a sketch-based picture conversion method, which includes:
obtaining a sketch to be converted and a sketch conversion instruction of a user;
determining a target picture type according to the sketch conversion instruction;
extracting the characteristics of the sketch to be converted to obtain target conversion characteristics of the sketch to be converted;
and carrying out picture conversion according to the target picture type and the target conversion characteristic to obtain a target conversion picture.
Optionally, the determining a target picture type according to the sketch conversion instruction includes:
determining a target label according to the sketch conversion instruction;
searching a picture type corresponding to the target label in a label type mapping table;
and determining the target picture type according to the picture type corresponding to the target label.
Optionally, the performing feature extraction on the sketch to be converted to obtain the target conversion feature of the sketch to be converted includes:
performing line recognition on the sketch to be converted to obtain line characteristics of the sketch to be converted;
carrying out color identification on the sketch to be converted to obtain the color characteristics of the sketch to be converted;
carrying out basic shape recognition on the sketch to be converted to obtain the shape characteristics of the sketch to be converted;
and determining the target conversion characteristics of the sketch to be converted according to the line characteristics, the color characteristics and the shape characteristics.
Optionally, the performing picture conversion according to the target picture type and the target conversion feature to obtain a target conversion picture includes:
determining a target conversion model according to the target picture type;
inputting the target conversion characteristics to the target conversion model for picture conversion to obtain an initial conversion picture and a picture matching degree corresponding to the initial conversion picture;
ranking the picture matching degree to obtain a ranking result;
and acquiring the matching degree of the first ranked target picture in the ranking result, and taking the initial conversion picture corresponding to the matching degree of the target picture as the target conversion picture.
Optionally, the determining a target conversion model according to the target picture type includes:
obtaining each sample picture type and a corresponding sample conversion picture;
determining a sample conversion sketch corresponding to the sample conversion picture according to the sample conversion picture;
performing model training on an initial conversion model according to the sample conversion pictures and the corresponding sample conversion sketch to obtain a sample conversion model corresponding to each sample picture type;
and determining a target conversion model according to the target picture type, the sample picture types and the sample conversion models corresponding to the sample picture types.
Optionally, the performing model training on the initial conversion model according to the sample conversion picture and the corresponding sample conversion sketch to obtain a sample conversion model corresponding to each sample picture type includes:
extracting features of the sample conversion sketch to obtain training line features, training color features and training shape features;
and inputting the training line characteristics, the training color characteristics, the training shape characteristics, the sample conversion pictures and the sample matching degrees corresponding to the sample conversion pictures into an initial conversion model for model training to obtain a sample conversion model corresponding to each sample picture type.
Optionally, after performing picture conversion according to the target picture type and the target conversion feature to obtain a target conversion picture, the method further includes:
sending the target conversion picture to the user, and receiving an evaluation result fed back by the user;
and returning to execute the step of performing picture conversion according to the target picture type and the target conversion characteristic to obtain a target conversion picture when the evaluation result is not the preset evaluation result.
In addition, to achieve the above object, the present invention further provides a sketch-based picture conversion apparatus, including:
the acquisition module is used for acquiring a sketch to be converted and a sketch conversion instruction of a user;
the determining module is used for determining the type of the target picture according to the sketch conversion instruction;
the extraction module is used for extracting the characteristics of the sketch to be converted to obtain the target conversion characteristics of the sketch to be converted;
and the conversion module is used for carrying out image conversion according to the target image type and the target conversion characteristic to obtain a target conversion image.
In addition, to achieve the above object, the present invention further provides a sketch-based picture conversion apparatus, including: a memory, a processor, and a sketch-based picture conversion program stored on the memory and executable on the processor, the sketch-based picture conversion program configured to implement the sketch-based picture conversion method as described above.
In addition, to achieve the above object, the present invention further provides a storage medium having a sketch-based picture conversion program stored thereon, wherein the sketch-based picture conversion program, when executed by a processor, implements the sketch-based picture conversion method as described above.
The method comprises the steps of obtaining a sketch to be converted and a sketch conversion instruction of a user; determining a target picture type according to the sketch conversion instruction; performing feature extraction on the sketch to be converted to obtain target conversion features of the sketch to be converted; and carrying out picture conversion according to the target picture type and the target conversion characteristic to obtain a target conversion picture. By the mode, the accurate target conversion characteristic and the target picture type are determined through the sketch to be converted and the sketch conversion instruction of the user, and picture conversion is carried out based on the target picture type and the target conversion characteristic, so that the accuracy and the high efficiency of picture conversion are ensured, and the experience of the user is improved.
Drawings
FIG. 1 is a schematic diagram of a sketch-based picture conversion device of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a first exemplary embodiment of a sketch-based picture conversion method according to the present invention;
FIG. 3 is a flowchart illustrating a second embodiment of a sketch-based picture conversion method according to the present invention;
FIG. 4 is a block diagram of a first embodiment of a sketch-based picture conversion device in accordance with the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
Referring to fig. 1, fig. 1 is a schematic structural diagram of a sketch-based picture conversion device in a hardware operating environment according to an embodiment of the present invention.
As shown in fig. 1, the sketch-based picture conversion apparatus may include: a processor 1001, such as a Central Processing Unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Wherein a communication bus 1002 is used to enable connective communication between these components. The user interface 1003 may include a Display (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a Wireless interface (e.g., a Wireless-Fidelity (Wi-Fi) interface). The Memory 1005 may be a Random Access Memory (RAM) Memory, or may be a Non-Volatile Memory (NVM), such as a disk Memory. The memory 1005 may alternatively be a storage device separate from the processor 1001 described previously.
Those skilled in the art will appreciate that the configuration shown in fig. 1 does not constitute a limitation of the sketch-based picture conversion apparatus, and may include more or less components than those shown, or combine certain components, or a different arrangement of components.
As shown in fig. 1, a memory 1005, which is a storage medium, may include therein an operating system, a network communication module, a user interface module, and a sketch-based picture conversion program.
In the sketch-based picture conversion apparatus shown in fig. 1, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 of the sketch-based picture conversion device according to the present invention may be disposed in a sketch-based picture conversion device, and the sketch-based picture conversion device calls a sketch-based picture conversion program stored in the memory 1005 through the processor 1001 and executes the sketch-based picture conversion method according to the embodiment of the present invention.
An embodiment of the present invention provides a sketch-based image conversion method, and referring to fig. 2, fig. 2 is a schematic flowchart of a first embodiment of a sketch-based image conversion method according to the present invention.
The sketch-based picture conversion method comprises the following steps:
step S10: and acquiring a sketch to be converted and a sketch conversion instruction of a user.
It should be noted that the execution main body of this embodiment is a terminal device, a sketch-based picture conversion system is installed on the terminal device, when the terminal device receives a sketch to be converted and a sketch conversion instruction sent by a user, the terminal device sends the sketch to be converted and the sketch conversion instruction to the sketch-based picture conversion system, the sketch-based picture conversion system determines a target picture type according to the sketch conversion instruction, performs feature extraction on the sketch to be converted to obtain a target conversion feature of the sketch to be converted, and finally performs picture conversion according to the target conversion feature and the target picture type to obtain a target conversion picture corresponding to the sketch to be converted. In this embodiment, the terminal device may be a computer, a mobile phone, or other intelligent terminal devices, which is not limited in this embodiment.
It is understood that the sketch to be converted refers to an image manually drawn by a user expressing the basic intention and concept of the user. The sketch conversion instruction refers to a conversion instruction which is sent by a user and used for carrying out an image conversion process based on the sketch conversion instruction and the sketch to be converted after receiving the sketch conversion instruction.
Step S20: and determining the type of the target picture according to the sketch conversion instruction.
It should be noted that the target picture type refers to a picture type of a sketch to be converted drawn by a user, for example, if the picture type of the sketch to be converted drawn by the user is a character image type, the target picture type is a character image type.
It can be understood that, in order to obtain an accurate target picture type according to the sketch conversion instruction, thereby ensuring the accuracy in the subsequent picture conversion, further, the determining the target picture type according to the sketch conversion instruction comprises determining a target label according to the sketch conversion instruction; searching a picture type corresponding to the target label in a label type mapping table; and determining the target picture type according to the picture type corresponding to the target label.
In a specific implementation, the target label of the type corresponding to the sketch to be converted is included in the sketch conversion instruction, but is not limited to the type corresponding to the sketch to be converted. The label type mapping table refers to a table for storing a mapping relationship between a picture type and a label after the picture type is labeled in advance. For example, the image type is labeled by letters, and a label corresponding to the image type of the old is a, a label corresponding to the image type of the building is B, and a label corresponding to the image type of the dog is C.
It should be noted that the target tag in the sketch conversion instruction is extracted, the picture type corresponding to the target tag is searched in the tag type mapping table, and the target picture type is determined according to the picture type corresponding to the target tag. For example, if the target tag in the sketch conversion instruction is a, and the image type corresponding to the target tag a is searched in the tag type mapping table and is the image type of the elderly, the target image type determined according to the sketch conversion instruction is the elderly.
Step S30: and extracting the characteristics of the sketch to be converted to obtain the target conversion characteristics of the sketch to be converted.
It should be noted that the target conversion feature refers to a feature extracted from the sketch to be converted and used for picture conversion. The target transformation features include, but are not limited to, line features, color features, and shape features of the sketch to be transformed. And after the draft to be converted is obtained, performing feature extraction on the draft to be converted so as to obtain the target conversion features of the draft to be converted.
It can be understood that, in order to extract an accurate target conversion feature based on a sketch to be converted, further, the performing feature extraction on the sketch to be converted to obtain the target conversion feature of the sketch to be converted includes: performing line recognition on the sketch to be converted to obtain line characteristics of the sketch to be converted; carrying out color identification on the sketch to be converted to obtain the color characteristics of the sketch to be converted; carrying out basic shape recognition on the sketch to be converted to obtain the shape characteristics of the sketch to be converted; and determining the target conversion characteristics of the sketch to be converted according to the line characteristics, the color characteristics and the shape characteristics.
In a specific implementation, after the sketch to be converted is obtained, line recognition is performed on the sketch to be converted to obtain all lines existing in the sketch to be converted and the outlines corresponding to the lines, and the line features of the sketch to be converted are obtained according to the lines and the outlines corresponding to the lines.
It should be noted that, color recognition is performed on the sketch to be converted, and color feature information related to the color in the sketch to be converted is extracted, so as to obtain the color feature in the sketch to be converted.
It can be understood that, the basic shape recognition is performed on the sketch to be converted, so as to obtain the basic shape in the sketch to be converted, and finally obtain the shape feature of the sketch to be converted. The base shapes include, but are not limited to, a series of base shapes such as rectangles, circles, ovals, and parallelograms.
In specific implementation, the method is determined according to the extracted line features, color features and shape features of the sketch to be recognized
Step S40: and carrying out picture conversion according to the target picture type and the target conversion characteristic to obtain a target conversion picture.
It should be noted that, after the target picture type and the target conversion feature are obtained, the sketch-based picture conversion system may perform picture analog conversion based on the target picture type and the target conversion feature, so as to target-convert the picture. The target conversion picture refers to a picture that the user wants to express and transmit corresponding to the sketch to be converted of the user, and the target conversion picture may be a picture that appears in the real world or a picture that does not appear in the real world, which is not limited in this embodiment.
It can be understood that, in order to enable the target conversion picture to meet the user requirement and further improve the user satisfaction, further, after the performing picture conversion according to the target picture type and the target conversion feature to obtain the target conversion picture, the method further includes: sending the target conversion picture to the user, and receiving an evaluation result fed back by the user; and returning to execute the step of performing picture conversion according to the target picture type and the target conversion characteristic to obtain a target conversion picture when the evaluation result is not the preset evaluation result.
In the specific implementation, the preset evaluation result refers to that the target conversion picture conforms to a picture corresponding to a sketch to be converted, after the target conversion picture is obtained, the target conversion picture is sent to a terminal where a user is located to be displayed, an evaluation result fed back by the user for the target conversion picture is received, when the evaluation result is the preset evaluation result, the target conversion picture is directly output, and when the evaluation result is that the target conversion picture does not conform to the picture corresponding to the sketch to be converted, the step of performing picture conversion according to the type of the target picture and the target conversion characteristic is returned to obtain the target conversion picture.
The method comprises the steps of obtaining a sketch to be converted and a sketch conversion instruction of a user; determining a target picture type according to the sketch conversion instruction; extracting the characteristics of the sketch to be converted to obtain target conversion characteristics of the sketch to be converted; and carrying out picture conversion according to the target picture type and the target conversion characteristic to obtain a target conversion picture. By the mode, the accurate target conversion characteristic and the target picture type are determined through the sketch to be converted and the sketch conversion instruction of the user, and picture conversion is carried out based on the target picture type and the target conversion characteristic, so that the accuracy and the high efficiency of picture conversion are ensured, and the experience of the user is improved.
Referring to fig. 3, fig. 3 is a flowchart illustrating a sketch-based picture conversion method according to a second embodiment of the present invention.
Based on the first embodiment, the step S40 in the sketch-based picture conversion method in this embodiment includes:
step S41: and determining a target conversion model according to the target picture type.
It should be noted that the target conversion model refers to a model obtained by deep learning training of a sample image type and a sample conversion sketch corresponding to the sample image type, and the target conversion model can perform image conversion based on target conversion characteristics, and output a plurality of conversion images corresponding to the sketch to be converted and image matching degrees corresponding to the conversion images. Since different image types correspond to different sample conversion models, after the target image type is obtained, the corresponding target conversion model needs to be determined in the sample conversion model according to the target image type.
It can be understood that, in order to obtain an accurate target conversion model, thereby ensuring that the converted target conversion picture highly matches the user requirements, further, each sample picture type and the corresponding sample conversion picture are obtained; determining a sample conversion sketch corresponding to the sample conversion picture according to the sample conversion picture; performing model training on an initial conversion model according to the sample conversion pictures and the corresponding sample conversion sketch to obtain a sample conversion model corresponding to each sample picture type; and determining a target conversion model according to the target picture type, the sample picture types and the sample conversion models corresponding to the sample picture types.
In a specific implementation, each sample picture type refers to multiple picture types stored in a preset database, for example, an old people picture type, a building picture type, a dog picture type, a child picture type, and the like, and may also be other picture types, which is not limited in this embodiment.
It should be noted that sample conversion pictures corresponding to each sample picture type are stored in the preset database, after the sample conversion pictures corresponding to the sample picture types are acquired, sample conversion sketches corresponding to each sample conversion picture are acquired in the preset database based on each sample conversion picture, each sample conversion sketch is a sample sketch drawn by a user for model training, one sample conversion sketch corresponds to multiple sample conversion pictures, each sample conversion picture is marked with a sample matching picture evaluated by the user for drawing the sample conversion sketch, and the sample matching pictures are used for embodying the matching degree between the sample conversion pictures and the sample conversion sketches.
It can be understood that after the sample conversion sketch and the sample conversion picture are obtained, model training is performed based on the sample conversion sketch and the sample conversion picture, so as to obtain a sample conversion model corresponding to each sample picture type. And matching according to the target picture type and the sample picture type, obtaining a sample conversion model corresponding to the sample picture type after finding the sample picture type same as the target picture type, and taking the sample conversion model corresponding to the sample picture type as a target conversion model.
In a specific implementation, in order to ensure accuracy of model training, further, the performing model training on the initial conversion model according to the sample conversion picture and the corresponding sample conversion sketch to obtain a sample conversion model corresponding to each sample picture type includes: performing feature extraction on the sample conversion sketch to obtain training line features, training color features and training shape features; and inputting the training line characteristics, the training color characteristics, the training shape characteristics, the sample conversion pictures and the sample matching degrees corresponding to the sample conversion pictures into an initial conversion model for model training to obtain a sample conversion model corresponding to each sample picture type.
It should be noted that, feature extraction is performed on the sample conversion sketch to obtain sample training features of the sample conversion sketch, where the sample training features include training line features, training color features, and training shape features, and the sample matching degrees corresponding to the training line features, the training color features, the training shape features, the sample conversion pictures, and the sample conversion pictures are input into the initial conversion model to perform model training, so as to obtain a sample conversion model corresponding to each sample picture type.
Step S42: and inputting the target conversion characteristics to the target conversion model for picture conversion to obtain an initial conversion picture and a picture matching degree corresponding to the initial conversion picture.
It should be noted that after the target conversion feature of the sketch to be converted is obtained, the target conversion feature is input into the target conversion model for image conversion, so as to obtain a plurality of initial conversion images corresponding to the sketch to be converted, and output the image matching degree of each initial conversion image and the sketch to be converted.
Step S43: and ranking the picture matching degree to obtain a ranking result.
It should be noted that, after the picture matching degree is obtained, the picture matching degree is ranked to obtain a ranking result of the picture matching degree corresponding to each initial conversion picture.
Step S44: and acquiring the matching degree of the first ranked target picture in the ranking result, and taking the initial conversion picture corresponding to the matching degree of the target picture as the target conversion picture.
It should be noted that after the ranking result is obtained, the picture matching degree ranked first in the picture matching degrees is obtained, the picture matching degree ranked first is taken as the target picture matching degree, and the initial conversion picture corresponding to the target picture matching degree is taken as the target conversion picture for picture output.
The embodiment determines a target conversion model according to the target picture type; inputting the target conversion characteristics to the target conversion model for picture conversion to obtain an initial conversion picture and a picture matching degree corresponding to the initial conversion picture; ranking the picture matching degree to obtain a ranking result; and acquiring the matching degree of the first ranked target picture in the ranking result, and taking the initial conversion picture corresponding to the matching degree of the target picture as the target conversion picture. By the mode, the image conversion is carried out based on the target conversion model and the target conversion characteristics corresponding to the target image type, the final target conversion image is obtained based on the image matching degree, and the accuracy of the image conversion and the customer satisfaction degree can be improved.
In addition, referring to fig. 4, an embodiment of the present invention further provides a sketch-based picture conversion apparatus, where the sketch-based picture conversion apparatus includes:
the obtaining module 10 is configured to obtain a sketch to be converted and a sketch conversion instruction of a user.
And the determining module 20 is configured to determine a target picture type according to the sketch conversion instruction.
And the extraction module 30 is configured to perform feature extraction on the sketch to be converted to obtain a target conversion feature of the sketch to be converted.
And the conversion module 40 is configured to perform picture conversion according to the target picture type and the target conversion characteristic to obtain a target conversion picture.
The method comprises the steps of obtaining a sketch to be converted and a sketch conversion instruction of a user; determining a target picture type according to the sketch conversion instruction; extracting the characteristics of the sketch to be converted to obtain target conversion characteristics of the sketch to be converted; and carrying out picture conversion according to the target picture type and the target conversion characteristic to obtain a target conversion picture. By the mode, the accurate target conversion characteristic and the target picture type are determined through the sketch to be converted and the sketch conversion instruction of the user, and picture conversion is carried out based on the target picture type and the target conversion characteristic, so that the accuracy and the high efficiency of picture conversion are ensured, and the experience of the user is improved.
In an embodiment, the determining module 20 is further configured to determine a target label according to the sketch conversion instruction;
searching a picture type corresponding to the target label in a label type mapping table;
and determining the target picture type according to the picture type corresponding to the target label.
In an embodiment, the extracting module 30 is further configured to perform line recognition on the sketch to be converted to obtain line features of the sketch to be converted;
carrying out color identification on the sketch to be converted to obtain the color characteristics of the sketch to be converted;
carrying out basic shape recognition on the sketch to be converted to obtain the shape characteristics of the sketch to be converted;
and determining the target conversion characteristics of the sketch to be converted according to the line characteristics, the color characteristics and the shape characteristics.
In an embodiment, the conversion module 40 is further configured to determine a target conversion model according to the target picture type;
inputting the target conversion characteristics to the target conversion model for picture conversion to obtain an initial conversion picture and a picture matching degree corresponding to the initial conversion picture;
ranking the picture matching degree to obtain a ranking result;
and acquiring the matching degree of the first ranked target picture in the ranking result, and taking the initial conversion picture corresponding to the matching degree of the target picture as the target conversion picture.
In an embodiment, the conversion module 40 is further configured to obtain each sample picture type and a corresponding sample conversion picture;
determining a sample conversion sketch corresponding to the sample conversion picture according to the sample conversion picture;
performing model training on an initial conversion model according to the sample conversion pictures and the corresponding sample conversion sketch to obtain a sample conversion model corresponding to each sample picture type;
and determining a target conversion model according to the target picture type, the sample picture types and the sample conversion models corresponding to the sample picture types.
In an embodiment, the conversion module 40 is further configured to perform feature extraction on the sample conversion sketch to obtain a training line feature, a training color feature, and a training shape feature;
and inputting the training line characteristics, the training color characteristics, the training shape characteristics, the sample conversion pictures and the sample matching degrees corresponding to the sample conversion pictures into an initial conversion model for model training to obtain a sample conversion model corresponding to each sample picture type.
In an embodiment, the conversion module 40 is further configured to send the target conversion picture to the user, and receive an evaluation result fed back by the user;
and returning to execute the step of performing picture conversion according to the target picture type and the target conversion characteristic to obtain a target conversion picture when the evaluation result is not the preset evaluation result.
Since the device adopts all technical solutions of all the embodiments, at least all the beneficial effects brought by the technical solutions of the embodiments are achieved, and no further description is given here.
Furthermore, an embodiment of the present invention further provides a storage medium, on which a sketch-based picture conversion program is stored, where the sketch-based picture conversion program, when executed by a processor, implements the steps of the sketch-based picture conversion method as described above.
Since the storage medium adopts all technical solutions of all the embodiments, at least all the beneficial effects brought by the technical solutions of the embodiments are achieved, and no further description is given here.
It should be noted that the above-described work flows are only exemplary, and do not limit the scope of the present invention, and in practical applications, a person skilled in the art may select some or all of them to achieve the purpose of the solution of the embodiment according to actual needs, and the present invention is not limited herein.
In addition, the technical details that are not described in detail in this embodiment may refer to the sketch-based image conversion method provided in any embodiment of the present invention, and are not described herein again.
Further, it is to be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention or portions thereof that contribute to the prior art may be embodied in the form of a software product, where the computer software product is stored in a storage medium (e.g. Read Only Memory (ROM)/RAM, magnetic disk, optical disk), and includes several instructions for enabling a terminal device (e.g. a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, and all equivalent structures or equivalent processes performed by the present invention or directly or indirectly applied to other related technical fields are also included in the scope of the present invention.
Claims (10)
1. A sketch-based picture conversion method is characterized by comprising the following steps:
obtaining a sketch to be converted and a sketch conversion instruction of a user;
determining a target picture type according to the sketch conversion instruction;
extracting the characteristics of the sketch to be converted to obtain target conversion characteristics of the sketch to be converted;
and carrying out picture conversion according to the target picture type and the target conversion characteristic to obtain a target conversion picture.
2. The sketch-based picture conversion method as claimed in claim 1, wherein said determining a target picture type according to said sketch conversion instruction comprises:
determining a target label according to the sketch conversion instruction;
searching a picture type corresponding to the target label in a label type mapping table;
and determining a target picture type according to the picture type corresponding to the target label.
3. The sketch-based picture conversion method as claimed in claim 1, wherein said extracting features of said sketch to be converted to obtain target conversion features of said sketch to be converted comprises:
performing line recognition on the sketch to be converted to obtain line characteristics of the sketch to be converted;
carrying out color identification on the sketch to be converted to obtain the color characteristics of the sketch to be converted;
carrying out basic shape recognition on the sketch to be converted to obtain the shape characteristics of the sketch to be converted;
and determining the target conversion characteristics of the sketch to be converted according to the line characteristics, the color characteristics and the shape characteristics.
4. The sketch-based picture conversion method as claimed in claim 1, wherein said performing picture conversion according to said target picture type and said target conversion characteristic to obtain a target conversion picture comprises:
determining a target conversion model according to the target picture type;
inputting the target conversion characteristics to the target conversion model for picture conversion to obtain an initial conversion picture and a picture matching degree corresponding to the initial conversion picture;
ranking the picture matching degree to obtain a ranking result;
and acquiring the matching degree of the first ranked target picture in the ranking result, and taking the initial conversion picture corresponding to the matching degree of the target picture as the target conversion picture.
5. The sketch-based picture conversion method of claim 4, wherein said determining a target conversion model according to said target picture type comprises:
acquiring each sample picture type and a corresponding sample conversion picture;
determining a sample conversion sketch corresponding to the sample conversion picture according to the sample conversion picture;
performing model training on an initial conversion model according to the sample conversion pictures and the corresponding sample conversion sketch to obtain a sample conversion model corresponding to each sample picture type;
and determining a target conversion model according to the target picture type, the sample picture types and the sample conversion models corresponding to the sample picture types.
6. The sketch-based picture conversion method as claimed in claim 5, wherein said model training of an initial conversion model according to said sample conversion picture and a corresponding sample conversion sketch to obtain a sample conversion model corresponding to each of said sample picture types comprises:
extracting features of the sample conversion sketch to obtain training line features, training color features and training shape features;
and inputting the training line characteristics, the training color characteristics, the training shape characteristics, the sample conversion pictures and the sample matching degrees corresponding to the sample conversion pictures into an initial conversion model for model training to obtain a sample conversion model corresponding to each sample picture type.
7. The sketch-based picture conversion method as claimed in any one of claims 1 to 6, wherein after performing picture conversion according to said target picture type and said target conversion feature to obtain a target conversion picture, further comprising:
sending the target conversion picture to the user, and receiving an evaluation result fed back by the user;
and when the evaluation result is not a preset evaluation result, returning to execute the step of performing picture conversion according to the target picture type and the target conversion characteristic to obtain a target conversion picture.
8. A sketch-based picture conversion apparatus, characterized in that the sketch-based picture conversion apparatus comprises:
the acquisition module is used for acquiring a sketch to be converted and a sketch conversion instruction of a user;
the determining module is used for determining the type of the target picture according to the sketch conversion instruction;
the extraction module is used for extracting the features of the sketch to be converted to obtain the target conversion features of the sketch to be converted;
and the conversion module is used for carrying out image conversion according to the target image type and the target conversion characteristic to obtain a target conversion image.
9. A sketch-based picture conversion apparatus, characterized in that the apparatus comprises: a memory, a processor, and a sketch-based picture conversion program stored on the memory and executable on the processor, the sketch-based picture conversion program being configured to implement the sketch-based picture conversion method as claimed in any one of claims 1 to 7.
10. A storage medium having stored thereon a sketch-based picture conversion program, which when executed by a processor implements the sketch-based picture conversion method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210319444.6A CN114757816A (en) | 2022-03-29 | 2022-03-29 | Sketch-based picture conversion method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210319444.6A CN114757816A (en) | 2022-03-29 | 2022-03-29 | Sketch-based picture conversion method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114757816A true CN114757816A (en) | 2022-07-15 |
Family
ID=82327487
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210319444.6A Pending CN114757816A (en) | 2022-03-29 | 2022-03-29 | Sketch-based picture conversion method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114757816A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060082571A1 (en) * | 2004-10-20 | 2006-04-20 | Siemens Technology-To-Business Center, Llc | Systems and methods for three-dimensional sketching |
| CN101478731A (en) * | 2008-12-25 | 2009-07-08 | 腾讯科技（深圳）有限公司 | Instant communication method, transmitting apparatus and system for mobile terminal |
| CN110163794A (en) * | 2018-05-02 | 2019-08-23 | 腾讯科技（深圳）有限公司 | Conversion method, device, storage medium and the electronic device of image |
| CN110599393A (en) * | 2019-09-06 | 2019-12-20 | 深圳前海微众银行股份有限公司 | Picture style conversion method, device and equipment and computer readable storage medium |
| CN111583408A (en) * | 2020-05-09 | 2020-08-25 | 中国科学技术大学 | Human body three-dimensional modeling system based on hand-drawn sketch |
| CN112106042A (en) * | 2018-05-29 | 2020-12-18 | 三星电子株式会社 | Electronic device and control method thereof |
| CN112927325A (en) * | 2021-03-22 | 2021-06-08 | 佛山市太火红鸟科技有限公司 | Commodity sketch generation method, device, equipment and storage medium |
-
2022
- 2022-03-29 CN CN202210319444.6A patent/CN114757816A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060082571A1 (en) * | 2004-10-20 | 2006-04-20 | Siemens Technology-To-Business Center, Llc | Systems and methods for three-dimensional sketching |
| CN101478731A (en) * | 2008-12-25 | 2009-07-08 | 腾讯科技（深圳）有限公司 | Instant communication method, transmitting apparatus and system for mobile terminal |
| CN110163794A (en) * | 2018-05-02 | 2019-08-23 | 腾讯科技（深圳）有限公司 | Conversion method, device, storage medium and the electronic device of image |
| CN112106042A (en) * | 2018-05-29 | 2020-12-18 | 三星电子株式会社 | Electronic device and control method thereof |
| CN110599393A (en) * | 2019-09-06 | 2019-12-20 | 深圳前海微众银行股份有限公司 | Picture style conversion method, device and equipment and computer readable storage medium |
| CN111583408A (en) * | 2020-05-09 | 2020-08-25 | 中国科学技术大学 | Human body three-dimensional modeling system based on hand-drawn sketch |
| CN112927325A (en) * | 2021-03-22 | 2021-06-08 | 佛山市太火红鸟科技有限公司 | Commodity sketch generation method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109727041B (en) | Intelligent customer service multi-turn question and answer method, equipment, storage medium and device | |
| CN111476227B (en) | Target field identification method and device based on OCR and storage medium | |
| JP7225548B2 (en) | Image processing device, image processing method and image processing program | |
| JP2020135852A (en) | Method, apparatus, electronic device, computer-readable storage medium, and computer program for image-based data processing | |
| CN110334179B (en) | Question-answer processing method, device, computer equipment and storage medium | |
| CN111401318B (en) | Action recognition method and device | |
| CN107679070B (en) | Intelligent reading recommendation method and device and electronic equipment | |
| CN110781284A (en) | Knowledge graph-based question and answer method, device and storage medium | |
| CN111027403A (en) | Gesture estimation method, device, equipment and computer readable storage medium | |
| CN111143530A (en) | Intelligent answering method and device | |
| CN111190595A (en) | Method, device, medium and electronic equipment for automatically generating interface code based on interface design drawing | |
| CN111046201A (en) | Garbage classification simulation system based on computer software | |
| JP2020057359A (en) | Training data generation method, training data generation apparatus, electronic device and computer-readable storage medium | |
| CN112231452A (en) | Question-answering method, device, equipment and storage medium based on natural language processing | |
| CN111859957A (en) | Method, device and equipment for extracting emotion reason clause labels and storage medium | |
| CN112381118B (en) | College dance examination evaluation method and device | |
| CN111638792A (en) | AR effect presentation method and device, computer equipment and storage medium | |
| CN114757816A (en) | Sketch-based picture conversion method, device, equipment and storage medium | |
| CN110362681B (en) | Method, device and storage medium for identifying repeated questions of question-answering system | |
| CN107784328B (en) | German old font identification method and device and computer readable storage medium | |
| CN114167993B (en) | Information processing method and device | |
| US10095802B2 (en) | Methods and systems for using field characteristics to index, search for, and retrieve forms | |
| CN112712450A (en) | Real-time interaction method, device, equipment and storage medium based on cloud classroom | |
| CN110163975B (en) | Method, device and equipment for drawing space straight line and storage medium | |
| CN111198926B (en) | Business handling management method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |