CN113688937A - Image processing method and device and storage medium - Google Patents
Image processing method and device and storage medium Download PDFInfo
- Publication number
- CN113688937A CN113688937A CN202111045756.4A CN202111045756A CN113688937A CN 113688937 A CN113688937 A CN 113688937A CN 202111045756 A CN202111045756 A CN 202111045756A CN 113688937 A CN113688937 A CN 113688937A
- Authority
- CN
- China
- Prior art keywords
- image
- trained
- cartoon
- initial
- conversion model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses an image processing method and device and a storage medium, wherein the image processing method comprises the following steps: acquiring an image to be trained and an animation image to be trained corresponding to the image to be trained; carrying out shielding treatment on the image to be trained to obtain a shielded image to be trained; training an initial cartoon image conversion model by using the image to be trained, the shielded image to be trained and the cartoon image to be trained to obtain a cartoon image conversion model; and under the condition of receiving the image to be processed, inputting the image to be processed into the cartoon image conversion model to obtain a cartoon image corresponding to the image to be processed.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image processing method and apparatus, and a storage medium.
Background
With the continuous development of the animation industry, users are more and more interested in transforming human images through artificial intelligence to obtain corresponding animation images.
In the prior art, a face image of a target object in a real image is detected and the face image is segmented, so that the segmented face image is subjected to animation processing, then a background in the real image is subjected to virtualization processing, so that an animation image corresponding to the real image is obtained, shadow or bright spots may exist in the face image due to illumination, and the shadow or bright spot area cannot be identified, so that the shadow or bright spot area cannot be converted into an animation area, and the quality of the animation image during animation of the image is reduced.
Disclosure of Invention
In order to solve the above technical problem, embodiments of the present application are directed to providing an image processing method and apparatus, and a storage medium, which can improve the quality of an animated image when the image is animated.
The technical scheme of the application is realized as follows:
the embodiment of the application provides an image processing method, which comprises the following steps:
acquiring an image to be trained and an animation image to be trained corresponding to the image to be trained;
shielding the image to be trained to obtain a shielded image to be trained;
training an initial cartoon image conversion model by using the image to be trained, the shielded image to be trained and the cartoon image to be trained to obtain a cartoon image conversion model;
and under the condition of receiving the image to be processed, inputting the image to be processed into the cartoon image conversion model to obtain a cartoon image corresponding to the image to be processed.
An embodiment of the present application provides an image processing apparatus, including:
the acquisition unit is used for acquiring an image to be trained and an animation image to be trained corresponding to the image to be trained;
the processing unit is used for shielding the image to be trained to obtain a shielded image to be trained;
the training unit is used for training an initial cartoon image conversion model by utilizing the image to be trained, the shielded image to be trained and the cartoon image to be trained to obtain a cartoon image conversion model;
and the input unit is used for inputting the image to be processed into the cartoon image conversion model under the condition of receiving the image to be processed to obtain the cartoon image corresponding to the image to be processed.
An embodiment of the present application provides an image processing apparatus, including:
the image processing system comprises a memory, a processor and a communication bus, wherein the memory is communicated with the processor through the communication bus, the memory stores an image processing program executable by the processor, and when the image processing program is executed, the processor executes the image processing method.
The embodiment of the application provides a storage medium, which stores a computer program, is applied to an image processing device, and is characterized in that the computer program realizes the image processing method when being executed by a processor.
The embodiment of the application provides an image processing method and device and a storage medium, wherein the image processing method comprises the following steps: acquiring an image to be trained and an animation image to be trained corresponding to the image to be trained; carrying out shielding treatment on the image to be trained to obtain a shielded image to be trained; training an initial cartoon image conversion model by using the image to be trained, the shielded image to be trained and the cartoon image to be trained to obtain a cartoon image conversion model; and under the condition of receiving the image to be processed, inputting the image to be processed into the cartoon image conversion model to obtain a cartoon image corresponding to the image to be processed. By adopting the method, the image processing device performs shielding processing on the acquired image to be trained to obtain the shielded image to be trained, so that the image to be trained, the shielded image to be trained and the animation image to be trained are utilized to train the initial animation image conversion model to obtain the animation image conversion model, the image processing device can identify the shielded area in the shielded image to be processed by utilizing the animation image conversion model to obtain the non-shielded image to be processed, and the non-shielded image to be processed is converted into the animation image, thereby improving the quality of the animation image during animation of the image.
Drawings
Fig. 1 is a flowchart of an image processing method according to an embodiment of the present application;
FIG. 2 is a first diagram illustrating an exemplary image processing provided by an embodiment of the present application;
fig. 3 is a schematic diagram of an exemplary image processing provided in an embodiment of the present application;
fig. 4 is a schematic diagram of exemplary image processing provided by an embodiment of the present application;
fig. 5 is a schematic diagram of an exemplary image processing provided by an embodiment of the present application;
fig. 6 is a first schematic structural diagram of an image processing apparatus according to an embodiment of the present disclosure;
fig. 7 is a schematic diagram illustrating a composition structure of an image processing apparatus according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Example one
An embodiment of the present application provides an image processing method, where the image processing method is applied to an image processing apparatus, and fig. 1 is a flowchart of the image processing method provided in the embodiment of the present application, and as shown in fig. 1, the image processing method may include:
s101, obtaining an image to be trained and an animation image to be trained corresponding to the image to be trained.
The image processing method is suitable for training the animation image conversion model by using the image to be trained, the animation image to be trained and the shielded image to be trained, so that the animation image conversion model is used for converting the image to be processed into the animation image in the scene.
In the embodiment of the present application, the image processing apparatus may be implemented in various forms. For example, the image processing devices described in the present application may include devices such as a mobile phone, a camera, a tablet computer, a notebook computer, a palmtop computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a navigation device, a wearable device, a smart band, a pedometer, and the like, as well as devices such as a Digital TV, a desktop computer, a server, and the like.
In the embodiment of the application, the image to be trained and the cartoon image to be trained can be images transmitted to the image processing device by other devices; the image to be trained and the cartoon image to be trained can also be images transmitted by the client and received by the image processing device; the image to be trained and the cartoon image to be trained can also be images stored in the image processing device; the image to be trained and the cartoon image to be trained can also be images acquired by the image processing device in other modes; the specific manner in which the image processing device acquires the image to be trained and the cartoon image to be trained can be determined according to actual conditions, which is not limited in the embodiments of the present application.
In the embodiment of the application, the number of the images to be trained is the same as that of the cartoon images to be trained.
It should be noted that the number of the images to be trained may be 1000; the number of the images to be trained can also be 6000; the number of the images to be trained can also be 10000; the number of the specific images to be trained can be determined according to actual conditions, which is not limited in the embodiment of the present application.
In the embodiment of the present application, the image to be trained may be a real image, such as a photo image. Specifically, the image to be trained can be a portrait photograph; the image to be trained can also be an animal photo; the image to be trained can also be a landscape photo; the image to be trained can also be other photo images; the specific image to be trained can be determined according to actual conditions, which is not limited in the embodiment of the present application.
It should be noted that the animation image to be trained is an animation image which is obtained according to the image to be trained and has a high similarity with the image to be trained. Specifically, the cartoon image to be trained can be a cartoon image artificially drawn according to the image to be trained; the cartoon image to be trained can also be a cartoon image drawn by a cartoon image drawing device according to the image to be trained; the cartoon image to be trained can also be a cartoon image obtained by carrying out cartoon conversion on the cartoon image to be trained by utilizing a cartoon image training model; the specific cartoon image to be trained can be determined according to actual conditions, and the embodiment of the application does not limit the specific cartoon image to be trained.
In this embodiment of the present application, a process of acquiring, by an image processing apparatus, an image to be trained and an animation image to be trained corresponding to the image to be trained includes: the image processing device acquires a first initial image to be trained, a first initial cartoon image to be trained corresponding to the first initial image to be trained and a second initial image to be trained; the image processing device trains an initial cartoon image training model by utilizing the first initial to-be-trained image and the first initial to-be-trained cartoon image to obtain a cartoon image training model; the image processing device inputs the second initial to-be-trained image into the cartoon image training model to obtain a second initial to-be-trained cartoon image corresponding to the second initial to-be-trained image; the image processing device takes the first initial cartoon image to be trained and the second initial cartoon image to be trained as cartoon images to be trained; and taking the first initial image to be trained and the second initial image to be trained as images to be trained.
It should be noted that the first initial cartoon image to be trained is a cartoon image drawn according to the first initial cartoon image to be trained. Specifically, the first initial cartoon image to be trained is a cartoon image which is drawn manually according to the first initial cartoon image to be trained and has high similarity with the first initial cartoon image to be trained.
Illustratively, as shown in FIG. 2: fig. 2 (a) is a real image, i.e., a first initial cartoon image to be trained, and fig. 2 (b) is a first initial cartoon image to be trained corresponding to the first initial cartoon image to be trained.
In the embodiment of the application, the first initial image to be trained is a partial image in the image to be trained, and the second initial image to be trained is an image except the first initial image to be trained in the image to be trained.
It should be noted that the number of the first initial images to be trained may be 1000; the number of the first initial images to be trained can also be 5000; the number of the first initial images to be trained can be other numbers; the specific number of the first initial images to be trained can be determined according to actual conditions, which is not limited in the embodiment of the present application.
It should be noted that the number of the second initial images to be trained may be 5000; the number of the second initial images to be trained can also be 8000; the number of the second initial images to be trained can be other numbers; the specific number of the second initial images to be trained may be determined according to an actual situation, which is not limited in the embodiment of the present application.
In the embodiment of the application, the image processing device comprises an initial cartoon image training model. Specifically, the initial animation image training model may be an existing Generative Adaptive Networks (GAN) pix2 pix; the initial cartoon image training model can also be other network models; the specific initial cartoon image training model can be determined according to actual conditions, and the embodiment of the application does not limit the model.
In the embodiment of the application, the information processing device trains the initial cartoon image training model by utilizing the first initial to-be-trained image and the first initial cartoon image to be trained to obtain the cartoon image training model with high animation quality and similarity to the initial to-be-trained image, and carries out animation conversion on the input second initial to-be-trained image by utilizing the cartoon image training model to obtain the second initial to-be-trained cartoon image with high animation quality and similarity to the second initial to-be-trained image.
Illustratively, as shown in FIG. 3: fig. 3 (a) is a real image, that is, a second initial animation image to be trained, and fig. 3 (b) is a second initial animation image to be trained, which is obtained by performing animation processing on the second initial image to be trained by using an animation image training model.
It can be understood that the cartoon image to be trained of the initial cartoon image conversion model is the cartoon image corresponding to the image to be trained, namely, the image to be trained and the cartoon image to be trained correspond one to one; the image processing device trains the initial cartoon image conversion model by utilizing the images to be trained and the cartoon images to be trained which have one-to-one correspondence relationship to obtain the cartoon image conversion model, so that the cartoon image obtained by converting the images to be processed by the cartoon image conversion model has high similarity with the images to be processed, and the accuracy of the image animation is improved.
It can be further understood that because the to-be-trained cartoon image and the to-be-trained image of the training initial cartoon image conversion model are both images with backgrounds, the obtained cartoon image conversion model can directly carry out cartoon conversion on the backgrounds in the to-be-processed images when the to-be-processed images are subjected to cartoon conversion, when the image processing device processes the to-be-processed images by using the cartoon image conversion model, the to-be-processed images do not need to be subjected to face detection, segmentation and other work, the cartoon effect of the faces and the backgrounds can be simultaneously completed, the processing speed of the image processing device when the to-be-processed images are processed by using the cartoon image conversion model is improved, and the quality of the cartoon images when the backgrounds of the to-be-processed images are subjected to cartoon conversion is also improved.
S102, carrying out shielding treatment on the image to be trained to obtain the shielded image to be trained.
In the embodiment of the application, after the image processing device acquires the image to be trained and the cartoon image to be trained corresponding to the image to be trained, the image processing device can shield the image to be trained to obtain the shielded image to be trained.
In the embodiment of the application, the mode of shielding the image to be trained by the image processing device comprises the steps of shielding the light spot of the image to be trained by the image processing device; shadow shielding processing can also be carried out on the image to be trained by the image processing device; or the image processing device carries out light spot shielding processing on the first position of the image to be trained and shadow shielding processing on the second position of the image to be trained; the specific way in which the image processing apparatus masks the image to be trained may be determined according to actual conditions, which is not limited in the embodiments of the present application.
In this embodiment of the application, the image processing apparatus performs occlusion processing on the image to be trained to obtain a process of obtaining an occluded image to be trained, including: the image processing device detects key points of an image to be trained to obtain key point coordinates; the image processing device determines a target area and/or a target contour according to the key point coordinates; the image processing device determines a target position in the target area and/or the target contour; and the image processing device generates a shielding image at the target position to obtain a shielded image to be trained.
In this embodiment of the application, the image processing apparatus performs the key point detection on the image to be trained, and the manner of obtaining the coordinates of the key points may be as follows: the image processing device can detect 68 key points of the image to be trained, and obtain the coordinates of the 68 key points.
In the embodiment of the application, if the image to be trained is a portrait image, the method of detecting key points of the image to be trained to obtain the coordinates of the key points may be a method of detecting key points of a human face of the image to be trained to obtain the coordinates of the key points.
In the embodiment of the application, if the image to be trained is a portrait image, the target region may be a human eye position region; or the target area is the area where the mouth is located; the target area may also be other location areas, and a specific target area may be determined according to an actual situation, which is not limited in this embodiment of the application.
In the embodiment of the application, if the image to be trained is a portrait image, the target contour may be a face contour; the target contour can also be a portrait contour; the target contour may also be other contours; the specific target profile can be determined according to actual conditions, and the embodiment of the application is not limited to this.
In the embodiment of the present application, the image processing apparatus may determine the target position in the target area and/or the target contour by determining the position of the eye in the target area and/or the target contour for the image processing apparatus, and taking the position as the target position; or randomly selecting a position in the target area and/or the target contour for the image processing device to obtain a target position; the position of the human face contour can be determined in the target area and/or the target contour for the image processing device, and the position of the human face contour is taken as the target position; the specific manner of determining the target position in the target area and/or the target contour by the image processing apparatus may be determined according to actual situations, which is not limited in the embodiment of the present application.
In this embodiment of the present application, a process of generating an occlusion image at a target position by an image processing apparatus to obtain an occluded image to be trained includes: the image processing device generates a shielding area at the target position according to the area generation rule; and the image processing device adjusts the brightness of the shielded area according to a preset brightness adjustment mode to obtain the shielded to-be-trained image.
In the embodiment of the present application, the region generation rule may be a rule configured in the image processing apparatus; the region generation rule may also be a rule received by the image processing apparatus before the image processing apparatus generates the occlusion region at the target position according to the region generation rule; the region generation rule may also be a rule acquired by the image processing apparatus in another manner, and a specific manner for acquiring the region generation rule by the image processing apparatus may be determined according to an actual situation, which is not limited in the embodiment of the present application.
In the embodiment of the present application, the region generation rule may be a rule for generating a polygonal smooth contour; the region generation rule may also be a rule for generating a circular smooth contour; the region generation rule may also be a rule that generates a smooth contour of an irregular shape; the specific region generation rule may be determined according to actual conditions, which is not limited in the embodiment of the present application.
In the embodiment of the present application, if the coordinates of the key points are the coordinates of 68 key points, the occlusion region may be a region including a plurality of key points in the 68 key points.
In the embodiment of the application, the image processing device adjusts the brightness of the shielded area according to the preset brightness adjustment mode, and may obtain the V channel information of the HSV channel corresponding to the image to be trained for the image processing device, and adjust the brightness information of the V channel corresponding to the shielded area in the image to be trained according to the preset brightness adjustment mode.
In this embodiment of the present application, the preset brightness adjustment manner may be a brightness adjustment manner configured in the image processing apparatus; the preset brightness adjustment mode can also be a brightness adjustment mode which is received by the image processing device and transmitted by other devices before the image processing device adjusts the brightness of the shielded area according to the preset brightness adjustment mode to obtain the shielded image to be trained; the preset brightness adjustment mode may also be a brightness adjustment mode acquired by the image processing apparatus in other manners, and the specific brightness adjustment mode acquired by the image processing apparatus may be determined according to an actual situation, which is not limited in this embodiment of the present application.
It should be noted that the preset brightness adjustment manner includes a preset bright spot brightness adjustment manner and a preset shadow brightness adjustment manner. For example, the preset bright spot brightness adjustment mode may be a brightness adjustment mode of +70 brightness of the shielded area; the preset shadow brightness adjustment mode can be a brightness adjustment mode of-50 of the brightness of the area to be shielded.
In this embodiment of the present application, the preset bright spot brightness adjustment manner may also be a manner of increasing the brightness of the shielding region by other amplitudes, and the specific preset bright spot brightness adjustment manner may be determined according to an actual situation, which is not limited in this embodiment of the present application. The preset shadow brightness adjustment mode may also be a mode of reducing the brightness of the occlusion region by other amplitudes, and the specific preset shadow brightness adjustment mode may be determined according to an actual situation, which is not limited in the embodiment of the present application.
Illustratively, as shown in fig. 4, a diagram (a) in fig. 4 is a real image, i.e., an image to be trained, a diagram (b) in fig. 4 is an image in which an eye generates a shadow mask, a diagram (c) in fig. 4 is an image in which a contour bright spot mask is generated at a chin, and a diagram (d) in fig. 4 is an animation image to be trained.
S103, training an initial cartoon image conversion model by using the image to be trained, the shielded image to be trained and the cartoon image to be trained to obtain a cartoon image conversion model.
In the embodiment of the application, the image processing device performs shielding processing on the image to be trained, and after the shielded image to be trained is obtained, the image processing device can train the initial cartoon image conversion model by using the image to be trained, the shielded image to be trained and the cartoon image to be trained, so as to obtain the cartoon image conversion model.
In this embodiment of the application, the process of obtaining the animation image conversion model by the image processing device training the initial animation image conversion model by using the image to be trained, the shielded image to be trained, and the animation image to be trained includes: the image processing device prunes the initial cartoon image conversion model to obtain a pruned initial cartoon image conversion model; and the image processing device obtains the cartoon image conversion model by utilizing the image to be trained, the shielded image to be trained and the initial cartoon image conversion model after training and pruning of the cartoon image to be trained.
In the embodiment of the application, the initial cartoon image conversion model can be a model built by using a similar GAN network; the initial cartoon image conversion model can also be other network models; the specific initial animation image conversion model can be determined according to actual conditions, and the embodiment of the application does not limit the model.
In the embodiment of the application, the image processing device may prune the initial cartoon image conversion model to obtain a pruned initial cartoon image conversion model; and then the image processing device trains the pruned initial cartoon image conversion model by utilizing the to-be-trained image, the shielded to-be-trained image and the to-be-trained cartoon image to obtain the cartoon image conversion model. The image processing device can also be used for training the initial cartoon image conversion model by utilizing the image to be trained, the shielded image to be trained and the cartoon image to be trained to obtain the trained initial cartoon image conversion model, and then the image processing device prunes the trained initial cartoon image conversion model to obtain the cartoon image conversion model. The specific manner in which the image processing apparatus obtains the animation image conversion model may be determined according to actual conditions, which is not limited in the embodiment of the present application.
In this embodiment of the application, in the process of pruning the initial cartoon image conversion model by the image processing device to obtain the pruned initial cartoon image conversion model, the inference speed of the pruned model may be determined after the initial cartoon image conversion model is pruned by the image processing device, and the image processing device determines that the pruned model is the pruned initial cartoon image conversion model if the inference speed of the pruned model meets the preset inference speed, otherwise, the image processing device continues pruning the pruned model until the inference speed of the pruned model meets the preset inference speed.
It should be noted that, in the case that the input image pixel to be trained is 256 × 256, the preset inference speed may be 40+ FPS, or the preset inference speed may be 70+ FPS, or the preset inference speed may be another speed value, which may be determined specifically according to the actual situation, and this is not limited in this embodiment of the present application.
It can be understood that the image processing device prunes the initial cartoon image conversion model by pruning the initial cartoon image conversion model, namely, reducing the number of convolution kernels in each convolution layer in the initial cartoon image conversion model and/or obtaining a little influence on a cartoon image in the initial cartoon image conversion model, so that the speed of the initial cartoon image conversion model when processing the image to be processed is increased, and the speed of the image processing device when converting the image to be processed into the cartoon image is increased.
And S104, under the condition that the image to be processed is received, inputting the image to be processed into the cartoon image conversion model to obtain a cartoon image corresponding to the image to be processed.
In the embodiment of the application, after the image processing device trains the initial cartoon image conversion model by using the image to be trained, the shielded image to be trained and the cartoon image to be trained to obtain the cartoon image conversion model, the image processing device can input the image to be processed into the cartoon image conversion model under the condition that the image processing device receives the image to be processed to obtain the cartoon image corresponding to the image to be processed.
Illustratively, as shown in fig. 5, a diagram (a) in fig. 5 is a real image, that is, an image to be processed, a shadow occlusion is formed at the right eye of the image to be processed due to a brim, and a diagram (b) in fig. 5 is an animation image obtained by performing animation transformation on the image to be processed by using an animation image transformation model, the shadow occlusion is correspondingly generated at the right eye of the animation image, and the right eye of the animation image still maintains a large-eye animation image.
The image processing device obtains the blocked image to be trained by blocking the obtained image to be trained, trains the initial cartoon image conversion model by utilizing the image to be trained, the blocked image to be trained and the cartoon image to be trained, obtains the cartoon image conversion model, enables the image processing device to identify the blocked area in the blocked image to be processed by utilizing the cartoon image conversion model, obtains the unblocked image to be processed, converts the unblocked image to be processed into the cartoon image, and improves the quality of the cartoon image during the animation of the image.
Example two
Based on the same inventive concept of the embodiments, the embodiments of the present application provide an image processing apparatus 1 corresponding to an image processing method; fig. 6 is a schematic diagram illustrating a first configuration of an image processing apparatus according to an embodiment of the present disclosure, where the image processing apparatus 1 may include:
the acquiring unit 11 is used for acquiring an image to be trained and an animation image to be trained corresponding to the image to be trained;
the processing unit 12 is configured to perform occlusion processing on the image to be trained to obtain an occluded image to be trained;
the training unit 13 is used for training an initial cartoon image conversion model by using the image to be trained, the shielded image to be trained and the cartoon image to be trained to obtain a cartoon image conversion model;
and the input unit 14 is configured to input the image to be processed into the animation image conversion model under the condition that the image to be processed is received, so as to obtain an animation image corresponding to the image to be processed.
In some embodiments of the present application, the apparatus further comprises a detection unit, a determination unit, and a generation unit;
the detection unit is used for detecting key points of the image to be trained to obtain key point coordinates;
the determining unit is used for determining a target area and/or a target outline according to the key point coordinates; determining a target position in the target area and/or target contour;
the generating unit is used for generating an occlusion image at the target position to obtain the occluded image to be trained.
In some embodiments of the present application, the apparatus further comprises an adjustment unit;
the generating unit is used for generating an occlusion region at the target position according to a region generating rule;
and the adjusting unit is used for adjusting the brightness of the shielded area according to a preset brightness adjusting mode to obtain the shielded image to be trained.
In some embodiments of the present application, the obtaining unit 11 is configured to obtain a first initial image to be trained, a first initial cartoon image to be trained corresponding to the first initial image to be trained, and a second initial image to be trained; the first initial cartoon image to be trained is a cartoon image drawn according to the first initial cartoon image to be trained;
the training unit 13 is configured to train an initial cartoon image training model by using the first initial to-be-trained image and the first initial to-be-trained cartoon image to obtain a cartoon image training model;
the input unit 14 is configured to input the second initial animation image to be trained into the animation image training model, so as to obtain a second initial animation image to be trained corresponding to the second initial animation image to be trained; taking the first initial cartoon image to be trained and the second initial cartoon image to be trained as the cartoon images to be trained; and taking the first initial image to be trained and the second initial image to be trained as the images to be trained.
In some embodiments of the present application, the apparatus further comprises a pruning unit;
the pruning unit is used for pruning the initial cartoon image conversion model to obtain a pruned initial cartoon image conversion model;
the training unit 13 is configured to train the clipped initial cartoon image conversion model by using the to-be-trained image, the blocked to-be-trained image, and the to-be-trained cartoon image, so as to obtain the cartoon image conversion model.
In practical applications, the obtaining Unit 11, the Processing Unit 12, the training Unit 13, and the input Unit 14 may be implemented by a processor 15 on the image Processing apparatus 1, specifically implemented by a CPU (Central Processing Unit), an MPU (micro processor Unit), a DSP (Digital Signal Processing), a Field Programmable Gate Array (FPGA), or the like; the above data storage may be realized by the memory 16 on the image processing apparatus 1.
An embodiment of the present application also provides an image processing apparatus 1, and as shown in fig. 7, the image processing apparatus 1 includes: a processor 15, a memory 16 and a communication bus 17, the memory 16 communicating with the processor 15 via the communication bus 17, the memory 16 storing a program executable by the processor 15, the program, when executed, performing the image processing method as described above via the processor 15.
In practical applications, the Memory 16 may be a volatile Memory (volatile Memory), such as a Random-Access Memory (RAM); or a non-volatile Memory (non-volatile Memory), such as a Read-Only Memory (ROM), a flash Memory (flash Memory), a Hard Disk (Hard Disk Drive, HDD) or a Solid-State Drive (SSD); or a combination of the above types of memories and provides instructions and data to the processor 15.
The present embodiment provides a computer-readable storage medium, on which a computer program is stored, which when executed by the processor 15 implements the image processing method as described above.
The image processing device obtains the blocked image to be trained by blocking the obtained image to be trained, trains the initial cartoon image conversion model by utilizing the image to be trained, the blocked image to be trained and the cartoon image to be trained, obtains the cartoon image conversion model, enables the image processing device to identify the blocked area in the blocked image to be processed by utilizing the cartoon image conversion model, obtains the unblocked image to be processed, converts the unblocked image to be processed into the cartoon image, and improves the quality of the cartoon image during the animation of the image.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present application, and is not intended to limit the scope of the present application.
Claims (10)
1. An image processing method, characterized in that the method comprises:
acquiring an image to be trained and an animation image to be trained corresponding to the image to be trained;
shielding the image to be trained to obtain a shielded image to be trained;
training an initial cartoon image conversion model by using the image to be trained, the shielded image to be trained and the cartoon image to be trained to obtain a cartoon image conversion model;
and under the condition of receiving the image to be processed, inputting the image to be processed into the cartoon image conversion model to obtain a cartoon image corresponding to the image to be processed.
2. The method according to claim 1, wherein the step of performing occlusion processing on the image to be trained to obtain an occluded image to be trained comprises:
detecting key points of the image to be trained to obtain key point coordinates;
determining a target area and/or a target contour according to the key point coordinates;
determining a target position in the target area and/or target contour;
and generating an occlusion image at the target position to obtain the occluded image to be trained.
3. The method of claim 2, wherein the generating an occlusion image at the target location, resulting in the occluded image to be trained, comprises:
generating an occlusion region at the target position according to a region generation rule;
and adjusting the brightness of the shielded area according to a preset brightness adjustment mode to obtain the shielded to-be-trained image.
4. The method according to claim 1, wherein the obtaining of the image to be trained and the cartoon image to be trained corresponding to the image to be trained comprises:
acquiring a first initial image to be trained, a first initial cartoon image to be trained corresponding to the first initial image to be trained and a second initial image to be trained; the first initial cartoon image to be trained is a cartoon image drawn according to the first initial cartoon image to be trained;
training an initial cartoon image training model by using the first initial to-be-trained image and the first initial to-be-trained cartoon image to obtain a cartoon image training model;
inputting the second initial to-be-trained image into the cartoon image training model to obtain a second initial to-be-trained cartoon image corresponding to the second initial to-be-trained image;
taking the first initial cartoon image to be trained and the second initial cartoon image to be trained as the cartoon images to be trained; and taking the first initial image to be trained and the second initial image to be trained as the images to be trained.
5. The method according to claim 1, wherein the training of an initial cartoon image conversion model by using the image to be trained, the blocked image to be trained and the cartoon image to be trained to obtain a cartoon image conversion model comprises:
pruning the initial cartoon image conversion model to obtain a pruned initial cartoon image conversion model;
and training the initial cartoon image conversion model after pruning by using the image to be trained, the shielded image to be trained and the cartoon image to be trained to obtain the cartoon image conversion model.
6. An image processing apparatus, characterized in that the apparatus comprises:
the acquisition unit is used for acquiring an image to be trained and an animation image to be trained corresponding to the image to be trained;
the processing unit is used for shielding the image to be trained to obtain a shielded image to be trained;
the training unit is used for training an initial cartoon image conversion model by utilizing the image to be trained, the shielded image to be trained and the cartoon image to be trained to obtain a cartoon image conversion model;
and the input unit is used for inputting the image to be processed into the cartoon image conversion model under the condition of receiving the image to be processed to obtain the cartoon image corresponding to the image to be processed.
7. The apparatus according to claim 6, further comprising a detection unit, a determination unit and a generation unit;
the detection unit is used for detecting key points of the image to be trained to obtain key point coordinates;
the determining unit is used for determining a target area and/or a target outline according to the key point coordinates; determining a target position in the target area and/or target contour;
the generating unit is used for generating an occlusion image at the target position to obtain the occluded image to be trained.
8. The apparatus of claim 7, further comprising an adjustment unit;
the generating unit is used for generating an occlusion region at the target position according to a region generating rule;
and the adjusting unit is used for adjusting the brightness of the shielded area according to a preset brightness adjusting mode to obtain the shielded image to be trained.
9. An image processing apparatus, characterized in that the apparatus comprises:
a memory, a processor, and a communication bus, the memory in communication with the processor through the communication bus, the memory storing a program of image processing executable by the processor, the program of image processing when executed performing the method of any of claims 1 to 5 by the processor.
10. A storage medium having stored thereon a computer program for application in an image processing apparatus, characterized in that the computer program, when being executed by a processor, implements the method of any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111045756.4A CN113688937A (en) | 2021-09-07 | 2021-09-07 | Image processing method and device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111045756.4A CN113688937A (en) | 2021-09-07 | 2021-09-07 | Image processing method and device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113688937A true CN113688937A (en) | 2021-11-23 |
Family
ID=78585614
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111045756.4A Pending CN113688937A (en) | 2021-09-07 | 2021-09-07 | Image processing method and device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113688937A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115100312A (en) * | 2022-07-14 | 2022-09-23 | 猫小兜动漫影视(深圳)有限公司 | Method and device for animating image |
-
2021
- 2021-09-07 CN CN202111045756.4A patent/CN113688937A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115100312A (en) * | 2022-07-14 | 2022-09-23 | 猫小兜动漫影视(深圳)有限公司 | Method and device for animating image |
| CN115100312B (en) * | 2022-07-14 | 2023-08-22 | 猫小兜动漫影视(深圳)有限公司 | Image cartoon method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102134405B1 (en) | System and Method for Improving Low Light Level Image Using Generative Adversarial Network | |
| US11948282B2 (en) | Image processing apparatus, image processing method, and storage medium for lighting processing on image using model data | |
| CN107771336B (en) | Feature detection and masking in images based on color distribution | |
| CN110503703B (en) | Method and apparatus for generating image | |
| US10410327B2 (en) | Shallow depth of field rendering | |
| CN112419170B (en) | Training method of shielding detection model and beautifying processing method of face image | |
| US20200234480A1 (en) | Systems and methods for realistic head turns and face animation synthesis on mobile device | |
| CN110781770B (en) | Living body detection method, device and equipment based on face recognition | |
| WO2022022043A1 (en) | Head image generation method, apparatus, server, and storage medium | |
| CN112330527A (en) | Image processing method, image processing apparatus, electronic device, and medium | |
| WO2020154000A1 (en) | Pose correction | |
| KR102128997B1 (en) | Apparatus, method and recording medium storing command for determining dental age | |
| CN113688937A (en) | Image processing method and device and storage medium | |
| CN110189252B (en) | Method and device for generating average face image | |
| WO2021056501A1 (en) | Feature point extraction method, movable platform and storage medium | |
| US10675955B2 (en) | Adaptive glare removal and/or color correction | |
| KR20230086996A (en) | A computer program that performs the noise addition process | |
| KR20230086998A (en) | A computer program that creates virtual character contnet by adding noise to face segment regions | |
| KR20230086999A (en) | A recording medium recording a virtual character content creation program | |
| CN111461971B (en) | Image processing method, device, equipment and computer readable storage medium | |
| CN111652807B (en) | Eye adjusting and live broadcasting method and device, electronic equipment and storage medium | |
| JP3578321B2 (en) | Image normalizer | |
| CN113920023A (en) | Image processing method and device, computer readable medium and electronic device | |
| CN111860372A (en) | Artificial intelligence-based expression package generation method, device, equipment and storage medium | |
| CN111199204A (en) | OpenGL-based face image processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |