CN114792285A - Image processing method and processing device, electronic device and readable storage medium - Google Patents

Image processing method and processing device, electronic device and readable storage medium Download PDF

Info

Publication number
CN114792285A
CN114792285A CN202210420012.4A CN202210420012A CN114792285A CN 114792285 A CN114792285 A CN 114792285A CN 202210420012 A CN202210420012 A CN 202210420012A CN 114792285 A CN114792285 A CN 114792285A
Authority
CN
China
Prior art keywords
image
area
training
target
target area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210420012.4A
Other languages
Chinese (zh)
Inventor
任帅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivo Mobile Communication Co Ltd
Original Assignee
Vivo Mobile Communication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivo Mobile Communication Co Ltd filed Critical Vivo Mobile Communication Co Ltd
Priority to CN202210420012.4A priority Critical patent/CN114792285A/en
Publication of CN114792285A publication Critical patent/CN114792285A/en
Priority to PCT/CN2023/088947 priority patent/WO2023202570A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G06T3/04—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G06T5/77—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Abstract

The application discloses an image processing method and processing device, an electronic device and a readable storage medium. An image processing method, comprising: positioning a target area in the first image to obtain position information of the target area; erasing image information of a preset area in the first image to obtain an erased second image, wherein the size of the second image is the same as that of the first image, and the preset area comprises a target area; according to the position information, intercepting an area image corresponding to the target area from the second image; and generating a processed target image according to the area image and the first image.

Description

Image processing method and processing device, electronic device and readable storage medium
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image processing method and apparatus, an electronic device, and a readable storage medium.
Background
In the related art, users sometimes need to erase or hide specific contents in the picture, such as erasing the text in the picture.
In the conventional processing method, for example, erasing a specific region by image restoration only utilizes texture information of the erased region, resulting in poor erasing effect and obvious restoration traces left on the picture. If the picture is processed by using the deep learning method, the repair traces can be reduced, but other regions close to the specific region in the picture are easy to erase together, and the processing effect is not good.
Disclosure of Invention
An object of the embodiments of the present application is to provide an image processing method and processing apparatus, an electronic device, and a readable storage medium, which can solve the problem of poor processing effect of erasing image content in the prior art.
In a first aspect, an embodiment of the present application provides an image processing method, including:
positioning a target area in the first image to obtain position information of the target area;
erasing image information of a preset area in a first image through an image processing model to obtain an erased second image, wherein the size of the second image is the same as that of the first image, and the preset area comprises a target area;
intercepting a region image corresponding to the target region in the second image according to the position information;
and generating a processed target image according to the area image and the first image.
In a second aspect, an embodiment of the present application provides an image processing apparatus, including:
the positioning module is used for positioning a target area in the first image to obtain the position information of the target area;
the erasing module is used for erasing the image information of a preset area in the first image through the image processing model to obtain an erased second image, wherein the size of the second image is the same as that of the first image, and the preset area comprises a target area;
the intercepting module is used for intercepting an area image corresponding to the target area in the second image according to the position information;
and the processing module is used for generating a processed target image according to the area image and the first image.
In a third aspect, embodiments of the present application provide an electronic device, including a processor and a memory, where the memory stores a program or instructions executable on the processor, and the program or instructions, when executed by the processor, implement the steps of the method according to the first aspect.
In a fourth aspect, embodiments of the present application provide a readable storage medium on which a program or instructions are stored, which when executed by a processor implement the steps of the method according to the first aspect.
In a fifth aspect, embodiments of the present application provide a chip, where the chip includes a processor and a communication interface, where the communication interface is coupled to the processor, and the processor is configured to execute a program or instructions to implement the steps of the method according to the first aspect.
In a sixth aspect, embodiments of the present application provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the method according to the first aspect.
In the embodiment of the present application, first, an image area to be processed is located, and position information is recorded. Then, the whole first image is subjected to image processing through a countermeasure model, the image processing step is based on an image recognition technology and a deep learning technology, the model automatically recognizes a part needing to be processed, a preset area needing to be processed is erased according to global information of the first image, and the image of the erased preset area is kept to be consistent with the whole image in a coordinated mode.
After the erasing is finished, according to the position information of the marked target area, the area image at the same position is intercepted from the second image after the erasing is finished through the image processing model, and the area image is combined with the original first image, namely only the part, which needs to be eliminated by a user, in the second image is selected to process the original image, so that the finally obtained target image erases the content of the target area needing to be processed under the condition that the coordination of the whole image is kept, meanwhile, other similar areas in the image are ensured not to be erased by mistake, and the processing efficiency when the specific content in the image is erased is improved.
Drawings
FIG. 1 shows one of the flow charts of an image processing method according to an embodiment of the present application;
FIG. 2 shows one of the schematic diagrams of an image processing method according to an embodiment of the application;
FIG. 3 shows a second schematic diagram of an image processing method according to an embodiment of the present application;
FIG. 4 is a third diagram of an image processing method according to an embodiment of the present application;
FIG. 5 shows a schematic structural diagram of an image processing model according to an embodiment of the present application;
FIG. 6 shows a second flowchart of an image processing method according to an embodiment of the present application;
fig. 7 is a block diagram showing the configuration of an image processing apparatus according to an embodiment of the present application;
FIG. 8 shows a block diagram of an electronic device according to an embodiment of the application;
fig. 9 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be described clearly below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, embodiments of the present application. All other embodiments, which can be derived from the embodiments in the present application by a person skilled in the art, are within the scope of protection of the present application.
The terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that embodiments of the application are capable of operation in sequences other than those illustrated or described herein, and that the terms "first," "second," etc. are generally used in a generic sense and do not limit the number of terms, e.g., a first term can be one or more than one. In addition, "and/or" in the specification and claims means at least one of connected objects, a character "/" generally means that a preceding and succeeding related objects are in an "or" relationship.
The image processing method and processing apparatus, the electronic device, and the readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings by specific embodiments and application scenarios thereof.
In some embodiments of the present application, there is provided an image processing method, and fig. 1 shows one of flowcharts of an image processing method according to an embodiment of the present application, and as shown in fig. 1, the method includes:
102, positioning a target area in a first image to obtain position information of the target area;
104, erasing image information of a preset area in the first image through an image processing model to obtain an erased second image;
in step 104, the size of the second image is the same as that of the first image, and the preset area comprises a target area;
106, intercepting a region image corresponding to the target region in the second image according to the position information;
and 108, generating a processed target image according to the area image and the first image.
In the embodiment of the present application, when a user wants to hide or erase a specific content in an image, for example, when the user wants to hide a text content in the image, first, a text area in the image may be located by, for example, a text recognition algorithm, and the location information corresponding to the text area is recorded. The position information may be coordinate information, among others.
After the position information of the target area needing to be erased is obtained, the whole original image of the first image is input into a preset image processing model, specifically, the image processing model can automatically recognize that the original image contains the target information needing to be erased in the first image according to the type of the information needing to be erased by a user, and the whole first image is processed according to the global information of the target image.
Specifically, fig. 2 shows one of schematic diagrams of an image processing method according to an embodiment of the present application, as shown in fig. 2, a first image 200 includes text information 202, and a user needs to erase the text information 202. First, by means such as Optical Character Recognition (OCR), the region where the text information 202 is located, that is, the target region 204, is marked in the first image 200, and the coordinate information of the target region 204 is recorded, thereby recording the position information of the target region 204.
And after the position information of the target area is obtained, inputting the first image into a preset image processing model. The image processing model can automatically identify the information content of the image, such as text information, which is to be erased by the user, and erase the information content based on the global information of the target image, including color information, pixel information and the like.
FIG. 3 is a second schematic diagram of an image processing method according to an embodiment of the present application, in which, as shown in FIG. 3, the image processing model processes the whole of the first image 300 and erases all of the predetermined regions identified as containing text. As shown in fig. 3, the image processing model performs erasure processing on 2 preset areas, where the first preset area 302 includes text information that the user needs to erase, and the second preset area 304 includes two-dimensional code information, and the two-dimensional code information is mistakenly identified as text and erased due to the close features of the two-dimensional code information and the text information.
Further, after the image processing model outputs the second image, the corresponding area image is cut out from the second image based on the same coordinates based on the position information of the identified target area, and the area image is superimposed on the original image of the first image according to the coordinates of the target area, thereby covering the target area of the first image and generating the target image.
FIG. 4 is a third schematic diagram of an image processing method according to an embodiment of the present application, as shown in FIG. 4, the text information in the target area 402 is erased and the two-dimensional code 404 is retained on the target image 400.
According to the embodiment of the application, the area image at the same position is intercepted from the second image erased by the image processing model according to the position information of the marked target area, and the area image is combined with the original first image, namely only the part, which needs to be eliminated by a user, in the second image is selected to process the original image, so that the finally obtained target image erases the content of the target area needing to be processed under the condition of keeping the coordination and consistency of the whole image, and meanwhile, other similar areas in the image are ensured not to be erased by mistake, and the processing efficiency when the specific content in the image is erased is improved.
In some embodiments of the present application, generating a processed target image from the region image and the first image comprises: and covering the area image in the target area according to the position information to obtain the target image.
In the embodiment of the present application, after the second image is obtained through the image processing model, the area image corresponding to the position of the target area is captured in the second image according to the position information of the target area, such as coordinate information.
After the area image is obtained, the intercepted area image processed by the image processing model is covered on the original first image according to the coordinate information of the target area, so that the image content in the target area is completely replaced by the area image, only the area needing to be erased in the replaced target image is replaced by the processed image, the content of the target area needing to be processed is erased under the condition that the coordination of the whole image is kept, meanwhile, other similar areas in the image are ensured not to be erased by mistake, and the processing efficiency when the specific content in the image is erased is improved.
In some embodiments of the application, before erasing the image information of the preset area in the first image, the method further comprises:
acquiring a first training image and a second training image, wherein the second training image is an image obtained by removing preset image information from the first training image;
training a preset model through a first training image and a second training image to obtain a trained image processing model, wherein the image processing model comprises a first network and a second network;
erasing image information of a preset area in a first image, comprising:
erasing the first image through a first network to obtain a processed third image;
performing downsampling processing on the third image to obtain a fourth image;
erasing the fourth image through a second network to obtain a processed fifth image;
and performing upsampling processing on the fifth image to obtain a second image.
In the embodiment of the application, a preset confrontation model is trained, so that a trained image processing model is obtained. Specifically, a first image is collected and collected first, and the first image is manually processed to generate a second image. The first image is an original image, and the second image is a second image obtained by erasing a preset area such as text content through software such as image editing or image modification.
And training a preset generation countermeasure network as a training set according to the first image and the second image, so as to erase the whole graph and character end to end. The method comprises the specific steps of building a network model, adopting a unet network structure for a generating network, firstly carrying out down-sampling on an image to obtain image semantic information, then carrying out up-sampling on the image to restore the image to an original size to obtain image output, and considering that a good result cannot be obtained by carrying out one-time model optimization on a complex scene, further processing a layer of light unet result after the unet (a dense prediction and segmentation U-shaped network) structure, and obtaining a final processing result.
Further, a discriminant network is constructed, wherein the discriminant network comprises a double-scale network, the first scale network comprises a plurality of cascaded convolutions, the step size of the convolution can be set to 1, and a pooling layer is not introduced, so that the resolution of the picture is not reduced.
And then adding a layer of same scale network after the first scale network, wherein the input of the second layer of scale network is obtained after the down sampling of the output of the first layer of scale network is doubled. The truth value (ground route) of the double-scale network is obtained by the erased picture and the picture which is sampled one time after the erased picture.
And inputting the generated network output result into a discrimination network, judging the difference between the currently generated erased picture and the originally marked erased picture (namely the second image) through the discrimination network, and using the difference as a loss (loss) back propagation optimization network parameter to finally obtain the optimized network structure.
In this optimized network structure, the model obtained by discriminating the network, i.e., the image processing model, is removed. Fig. 5 shows a schematic structural diagram of an image processing model according to an embodiment of the present application, and as shown in fig. 5, the image processing model 500 includes a 2-layer unet network structure, i.e., a first network 502 and a second network 504.
When erasing specific content in the first image 506 through the image processing model 500, first, the first image 506 to be processed is input into the first network 502, and specific information therein is erased through the first network 502.
The third image 508 after the erasure is obtained after the processing, the third image 508 is down-sampled to obtain a fourth image 510 with a reduced resolution, so as to obtain an image semantic definition, and the fourth image 510 obtained by down-sampling is used as an input of a second network, that is, the second network 504.
Finally, the fifth image 512 output by the second network 504 is up-sampled and restored to the original size to obtain the final second image 514.
It can be understood that the content contained in the first image, i.e., the content that the user needs to erase. If the user needs to erase the character information in the image, the first image comprises the character information. If the user needs to erase the face, the first image contains face information.
By training the image recognition model, the corresponding content area in the first image can be erased by the image recognition model according to the image content in the training set, the erased image can keep the coordination of the whole image, and the image processing efficiency is improved.
In some embodiments of the present application, the target area is a character image area, and locating the target area in the first image includes:
carrying out optical character recognition on the first image, and acquiring a character detection frame in the first image;
and positioning the target area in the first image according to the coordinate information of the character detection frame.
In the embodiment of the present application, the target area specifically includes a character image area, that is, the user needs to erase the character area in the first image. Specifically, first, Optical Character (OCR) recognition is performed on the first image, so that the positions of characters are detected in the first image, and a character detection frame is formed based on the positions of the characters.
Specifically, the first image is first preprocessed, and in some embodiments, a denoising algorithm is used to remove noise from the first image. And then, acquiring a text or a character through an OCR detection algorithm according to the trained OCR detection model, and positioning coordinate information, wherein the OCR detection algorithm can acquire character detection frames in various scenes such as horizontal, vertical and bending scenes.
For the horizontal and vertical regular character detection frames, a four-point coordinate frame can be adopted for representation; and for a curved irregular character detection box, an eight-point coordinate box can be used for representation. And if the current picture does not have the character information or the character information, returning to an empty character coordinate frame.
The character detection frame is marked with coordinate information that refers to coordinates of the character detection frame in the first image. The target area is positioned in the first image through the coordinate information of the character detection frame, so that after the second image is obtained through the image processing model, the processed area image is covered on the target area according to the coordinate information, the generated target image is enabled to ensure the coordination of the whole image, effectively erase the content of the target area needing to be processed, simultaneously ensure that other similar areas in the image cannot be erased by mistake, and improve the processing efficiency when the specific content in the image is erased.
In some embodiments of the present application, fig. 6 shows a second flowchart of an image processing method according to an embodiment of the present application, and as shown in fig. 6, the method includes:
step 602, performing text positioning on an original picture, and recording character coordinate information in the original picture;
in step 602, the image is first preprocessed, and a denoising algorithm is used to remove noise from the image. Then, a detection model is trained ocr, and text positioning information is acquired through a ocr detection algorithm, and the ocr detection algorithm can acquire text detection boxes in various scenes such as horizontal, vertical and bending. Four-point coordinate boxes are used for horizontal and vertical text boxes, and eight-point coordinate boxes are used for curved text boxes. And if the current picture does not have the character information, returning to an empty text coordinate box.
Step 604, collecting paired data, training and generating a confrontation network for erasing characters;
in step 604, the paired < original picture, character erasure picture > is collected, and model training is performed in a generation countermeasure mode to obtain a picture processing model.
Specifically, an original picture is acquired and obtained first, and a corresponding character erasure picture is acquired through a PS, so that paired < original picture, character erasure picture > are obtained.
Then training a generation countermeasure network to erase the whole graph and text end to end; the method specifically comprises the steps of building a network model, adopting a unet network structure for a generation network, firstly carrying out down-sampling on an image to obtain image semantic information, then carrying out up-sampling on the image to restore the image to an original size to obtain image output, and considering that a good result cannot be obtained by one-time model optimization on a complex scene, further processing a layer of light unet behind the unet structure to obtain a final processing result. And constructing a discrimination network which consists of a double-scale network, wherein the first scale network consists of a plurality of convolution cascades, the step length of the convolution is set to be 1, a pooling layer is not introduced, the resolution of the picture is ensured not to be reduced, then, a layer of the same scale network is added behind the first scale network, but the input of the second layer of scale network is obtained by sampling the output of the first layer of network by one time. And the ground route of the double-scale network is obtained by sampling the erased picture and the erased picture by one time respectively.
And inputting the generated network output result into a discrimination network to judge the difference between the currently generated erased picture and the originally marked erased picture, and using the difference as a loss back propagation optimization network parameter to finally obtain an optimized network structure.
Step 606, reasoning the original picture by adopting the optimized model to obtain an erased picture result;
in step 606, the optimized generation countermeasure model is used to reason the original picture to obtain the erased picture result, specifically, the original picture is input into the trained model, at this time, the discriminator needs to be removed, only the generator needs to be reserved, and the output picture obtained after the generator is the erased picture result.
Step 608, mapping the acquired character coordinate information to the erased picture, and cutting out an erasing area where the text coordinate is located;
in step 608, mapping the obtained character coordinate information to the obtained erased picture result, and cutting out an erased area where the text coordinate is located; considering that some areas similar to characters, such as fences, lines, flowers and plants, are easy to erase as characters, therefore, in order to ensure that the information is not mistakenly smeared, character coordinate information recorded from an original picture needs to be mapped onto an acquired erasing picture, because coordinates are non-rectangular frames, a mask picture as large as an input picture needs to be set, the mask picture is originally pure black, and the area where the coordinate frame is located is white, so that the erasing area where the text coordinates needing to be cut out are located can be obtained.
And step 610, pasting the erasing area back to the original image to obtain a final character erasing picture.
In step 610, the clipped erasure area is pasted back to the original image, so as to obtain the final erasure image.
The method comprises the specific steps that when a mask picture corresponding to a character area needing to be erased is obtained, only an erasing area corresponding to a pure white area in the mask picture needs to be attached to an original picture, and therefore the final required character erasing picture can be obtained, and mistaken smearing of an area similar to characters is avoided.
In some embodiments of the present application, an image processing apparatus is provided, and fig. 7 shows a block diagram of a structure of the image processing apparatus according to an embodiment of the present application, and as shown in fig. 7, an image processing apparatus 700 includes:
a positioning module 702, configured to position a target area in a first image to obtain location information of the target area;
an erasing module 704, configured to erase image information of a preset region in a first image through an image processing model to obtain an erased second image, where a size of the second image is the same as that of the first image, and the preset region includes a target region;
an intercepting module 706, configured to intercept, according to the position information, an area image corresponding to the target area in the second image;
and a processing module 708, configured to generate a processed target image according to the region image and the first image.
According to the embodiment of the application, the area image at the same position is intercepted from the second image erased by the image processing model according to the position information of the marked target area, and the area image is combined with the original first image, namely only the part, which needs to be eliminated by a user, in the second image is selected to process the original image, so that the finally obtained target image erases the content of the target area needing to be processed under the condition that the coordination and the consistency of the whole image are kept, meanwhile, other similar areas in the image are ensured not to be erased by mistake, and the processing efficiency when the specific content in the image is erased is improved.
In some embodiments of the present application, the image processing apparatus further comprises: and the covering module is used for covering the area image in the target area according to the position information to obtain the target image.
After the area image is obtained, the intercepted area image processed by the image processing model is covered on the original first image according to the coordinate information of the target area, so that the image content in the target area is completely replaced by the area image, only the area needing to be erased in the replaced target image is replaced by the processed image, the content of the target area needing to be processed is erased under the condition that the coordination and consistency of the whole image are kept, meanwhile, other similar areas in the image are ensured not to be erased by mistake, and the processing efficiency in erasing the specific content in the image is improved.
In some embodiments of the present application, the processing device further comprises:
the acquisition module is used for acquiring a first training image and a second training image, wherein the second training image is an image obtained by removing preset image information from the first training image;
the training module is used for training a preset model through a first training image and a second training image to obtain a trained image processing model, and the image processing model comprises a first network and a second network;
the erasing module is further used for erasing the first image through a first network to obtain a processed third image;
the sampling module is used for carrying out downsampling processing on the third image to obtain a fourth image;
the erasing module is further used for erasing the fourth image through a second network to obtain a processed fifth image;
and the sampling module is also used for carrying out up-sampling processing on the fifth image to obtain a second image.
By training the image recognition model, the corresponding content area in the first image can be erased by the image recognition model according to the image content in the training set, the erased image can keep the coordination of the whole image, and the image processing efficiency is improved.
In some embodiments of the application, the target area is a character image area, and the processing device further includes:
the recognition module is used for carrying out optical character recognition on the first image and acquiring a character detection frame in the first image;
and the positioning module is also used for positioning the target area in the first image according to the coordinate information of the character detection frame.
According to the method and the device, the target area is positioned in the first image, so that after the second image is obtained through the image processing model, the processed area image is covered on the target area according to the coordinate information, the generated target image is enabled to be consistent in overall image, the content of the target area needing to be processed is effectively erased, meanwhile, other similar areas in the image are enabled not to be erased by mistake, and the processing efficiency when the specific content in the image is erased is improved.
The image processing apparatus in the embodiment of the present application may be an electronic device, or may be a component in an electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal, or may be a device other than a terminal. The electronic Device may be, for example, a Mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic Device, a Mobile Internet Device (MID), an Augmented Reality (AR)/Virtual Reality (VR) Device, a robot, a wearable Device, an ultra-Mobile personal computer (UMPC), a netbook or a Personal Digital Assistant (PDA), and the like, and may also be a server, a Network Attached Storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), an assistant, or a self-service machine, and the embodiments of the present application are not limited in particular.
The image processing apparatus in the embodiment of the present application may be an apparatus having an operating system. The operating system may be an Android operating system (Android), an iOS operating system, or other possible operating systems, which is not specifically limited in the embodiments of the present application.
The image processing apparatus provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and is not described here again to avoid repetition.
Optionally, an electronic device is further provided in an embodiment of the present application, fig. 8 shows a block diagram of a structure of the electronic device according to the embodiment of the present application, and as shown in fig. 8, an electronic device 800 includes a processor 802, a memory 804, and a program or an instruction stored in the memory 804 and capable of running on the processor 802, and when the program or the instruction is executed by the processor 802, the process of the embodiment of the method is implemented, and the same technical effect can be achieved, and details are not repeated here to avoid repetition.
It should be noted that the electronic device in the embodiment of the present application includes the mobile electronic device and the non-mobile electronic device described above.
Fig. 9 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
The electronic device 900 includes, but is not limited to: a radio frequency unit 901, a network module 902, an audio output unit 903, an input unit 904, a sensor 905, a display unit 906, a user input unit 907, an interface unit 908, a memory 909, and a processor 910.
Those skilled in the art will appreciate that the electronic device 900 may further comprise a power supply (e.g., a battery) for supplying power to various components, and the power supply may be logically connected to the processor 910 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system. The electronic device structure shown in fig. 9 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than those shown, or combine some components, or arrange different components, and thus, the description is not repeated here.
The processor 910 is configured to locate a target area in the first image, and obtain location information of the target area;
erasing image information of a preset area in a first image through an image processing model to obtain an erased second image, wherein the size of the second image is the same as that of the first image, and the preset area comprises a target area;
intercepting a region image corresponding to the target region in the second image according to the position information;
and generating a processed target image according to the area image and the first image.
According to the embodiment of the application, the area image at the same position is intercepted from the second image erased by the image processing model according to the position information of the marked target area, and the area image is combined with the original first image, namely only the part, which needs to be eliminated by a user, in the second image is selected to process the original image, so that the finally obtained target image erases the content of the target area needing to be processed under the condition that the coordination and the consistency of the whole image are kept, meanwhile, other similar areas in the image are ensured not to be erased by mistake, and the processing efficiency when the specific content in the image is erased is improved.
Optionally, the processor 910 is further configured to cover the region image in the target region according to the position information, so as to obtain a target image.
According to the embodiment of the application, after the area image is obtained, the intercepted area image processed by the image processing model is covered on the original first image according to the coordinate information of the target area, so that the image content in the target area is completely replaced by the area image, only the area needing to be erased in the replaced target image is replaced by the processed image, the content of the target area needing to be processed is erased under the condition that the coordination of the whole image is kept, meanwhile, other similar areas in the image are prevented from being erased by mistake, and the processing efficiency in erasing the specific content in the image is improved.
Optionally, the processor 910 is further configured to obtain a first training image and a second training image, where the second training image is image data obtained by erasing a preset region in the first training image;
training a preset model through a first training image and a second training image to obtain a trained image processing model, wherein the image processing model comprises a first network and a second network;
erasing image information of a preset area in a first image, comprising:
erasing the first image through a first network to obtain a processed third image;
performing downsampling processing on the third image to obtain a fourth image;
erasing the fourth image through a second network to obtain a processed fifth image;
and performing upsampling processing on the fifth image to obtain a second image.
By training the image recognition model, the corresponding content area in the first image can be erased by the image recognition model according to the image content in the training set, the erased image can keep the coordination of the whole image, and the image processing efficiency is improved.
Optionally, the target area is a character image area, and the processor 910 is further configured to perform optical character recognition on the first image, and obtain a character detection frame in the first image;
and positioning the target area in the first image according to the coordinate information of the character detection frame.
According to the method and the device, the target area is positioned in the first image, so that after the second image is obtained through the image processing model, the processed area image is covered on the target area according to the coordinate information, the generated target image is enabled to ensure the coordination of the whole image, the content of the target area needing to be processed is effectively erased, other similar areas in the image are ensured not to be erased by mistake, and the processing efficiency when the specific content in the image is erased is improved.
It should be understood that, in the embodiment of the present application, the input Unit 904 may include a Graphics Processing Unit (GPU) 9041 and a microphone 9042, and the Graphics processor 9041 processes image data of a still picture or a video obtained by an image capturing device (such as a camera) in a video capturing mode or an image capturing mode. The display unit 906 may include a display panel 9061, and the display panel 9061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 907 includes at least one of a touch panel 9071 and other input devices 9072. A touch panel 9071 also referred to as a touch screen. The touch panel 9071 may include two parts, a touch detection device and a touch controller. Other input devices 9072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein.
The memory 909 may be used to store software programs as well as various data. The memory 909 may mainly include a first storage area storing a program or an instruction and a second storage area storing data, wherein the first storage area may store an operating system, an application program or an instruction (such as a sound playing function, an image playing function, and the like) required for at least one function, and the like. Further, the memory 909 may include volatile memory or nonvolatile memory, or the memory 909 may include both volatile and nonvolatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. The volatile Memory may be a Random Access Memory (RAM), a Static Random Access Memory (Static RAM, SRAM), a Dynamic Random Access Memory (Dynamic RAM, DRAM), a Synchronous Dynamic Random Access Memory (Synchronous DRAM, SDRAM), a Double Data Rate Synchronous Dynamic Random Access Memory (Double Data Rate SDRAM, ddr SDRAM), an Enhanced Synchronous SDRAM (ESDRAM), a Synchronous Link DRAM (SLDRAM), and a Direct Memory bus RAM (DRRAM). The memory 909 in the embodiments of the subject application includes, but is not limited to, these and any other suitable types of memory.
Processor 910 may include one or more processing units; optionally, the processor 910 integrates an application processor, which mainly handles operations related to the operating system, user interface, and applications, and a modem processor, which mainly handles wireless communication signals, such as a baseband processor. It is to be appreciated that the modem processor described above may not be integrated into processor 910.
The embodiment of the present application further provides a readable storage medium, where a program or an instruction is stored on the readable storage medium, and when the program or the instruction is executed by a processor, the program or the instruction implements the processes of the foregoing method embodiments, and can achieve the same technical effects, and in order to avoid repetition, details are not repeated here.
The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a Read-only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and so on.
The embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to execute a program or an instruction to implement each process of the foregoing method embodiments, and can achieve the same technical effect, and in order to avoid repetition, the details are not repeated here.
It should be understood that the chips mentioned in the embodiments of the present application may also be referred to as a system-on-chip, or a system-on-chip.
Embodiments of the present application provide a computer program product, where the program product is stored in a storage medium, and the program product is executed by at least one processor to implement the processes of the foregoing method embodiments, and can achieve the same technical effects, and in order to avoid repetition, details are not described here again.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Further, it should be noted that the scope of the methods and apparatuses in the embodiments of the present application is not limited to performing the functions in the order illustrated or discussed, but may include performing the functions in a substantially simultaneous manner or in a reverse order based on the functions recited, e.g., the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.
Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method of the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solutions of the present application or portions thereof that contribute to the prior art may be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) to execute the method according to the embodiments of the present application.
While the present embodiments have been described with reference to the accompanying drawings, it is to be understood that the present embodiments are not limited to those precise embodiments, which are intended to be illustrative rather than restrictive, and that various changes and modifications may be effected therein by one skilled in the art without departing from the scope of the appended claims.

Claims (10)

1. An image processing method, characterized by comprising:
positioning a target area in a first image to obtain position information of the target area;
erasing image information of a preset area in the first image to obtain an erased second image, wherein the size of the second image is the same as that of the first image, and the preset area comprises the target area;
intercepting a region image corresponding to the target region in the second image according to the position information;
and generating a processed target image according to the area image and the first image.
2. The image processing method according to claim 1, wherein the generating a processed target image from the region image and the first image comprises:
and covering the area image in the target area according to the position information to obtain the target image.
3. The image processing method according to claim 1, wherein before said erasing image information of a preset area in the first image, the method further comprises:
acquiring a first training image and a second training image, wherein the first training image comprises preset image information, and the second training image is an image obtained by removing the preset image information from the first training image;
training a preset model through the first training image and the second training image to obtain a trained image processing model, wherein the image processing model comprises a first network and a second network;
the erasing of the image information of the preset area in the first image includes:
erasing the first image through the first network to obtain a processed third image;
performing downsampling processing on the third image to obtain a fourth image;
erasing the fourth image through the second network to obtain a processed fifth image;
and performing upsampling processing on the fifth image to obtain the second image.
4. The image processing method according to any one of claims 1 to 3, wherein the target area is a character image area, and the positioning the target area in the first image includes:
carrying out optical character recognition on the first image, and acquiring a character detection frame in the first image;
and positioning the target area in the first image according to the coordinate information of the character detection frame.
5. An image processing apparatus characterized by comprising:
the positioning module is used for positioning a target area in a first image to obtain the position information of the target area;
the erasing module is used for erasing image information of a preset area in the first image to obtain an erased second image, wherein the size of the second image is the same as that of the first image, and the preset area comprises the target area;
an intercepting module, configured to intercept, according to the position information, an area image corresponding to the target area in the second image;
and the processing module is used for generating a processed target image according to the area image and the first image.
6. The image processing apparatus according to claim 5, characterized by further comprising:
and the covering module is used for covering the area image in the target area according to the position information to obtain the target image.
7. The image processing apparatus according to claim 5, further comprising:
the acquisition module is used for acquiring a first training image and a second training image, wherein the second training image is image data obtained after the preset area in the first training image is erased;
the training module is used for training a preset model through the first training image and the second training image to obtain the trained image processing model, and the image processing model comprises a first network and a second network;
the erasing module is further configured to erase the first image through the first network to obtain a processed third image;
the sampling module is used for carrying out downsampling processing on the third image to obtain a fourth image;
the erasing module is further configured to erase the fourth image through the second network to obtain a processed fifth image;
the sampling module is further configured to perform upsampling processing on the fifth image to obtain the second image.
8. The image processing apparatus according to any one of claims 5 to 7, wherein the target region is a character image region, the processing apparatus further comprising:
the recognition module is used for carrying out optical character recognition on the first image and acquiring a character detection frame in the first image;
the positioning module is further configured to position the target area in the first image according to the coordinate information of the character detection box.
9. An electronic device, comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the image processing method of any one of claims 1 to 4.
10. A readable storage medium, characterized in that it stores thereon a program or instructions which, when executed by a processor, implement the steps of the image processing method according to any one of claims 1 to 4.
CN202210420012.4A 2022-04-21 2022-04-21 Image processing method and processing device, electronic device and readable storage medium Pending CN114792285A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202210420012.4A CN114792285A (en) 2022-04-21 2022-04-21 Image processing method and processing device, electronic device and readable storage medium
PCT/CN2023/088947 WO2023202570A1 (en) 2022-04-21 2023-04-18 Image processing method and processing apparatus, electronic device and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210420012.4A CN114792285A (en) 2022-04-21 2022-04-21 Image processing method and processing device, electronic device and readable storage medium

Publications (1)

Publication Number Publication Date
CN114792285A true CN114792285A (en) 2022-07-26

Family

ID=82461312

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210420012.4A Pending CN114792285A (en) 2022-04-21 2022-04-21 Image processing method and processing device, electronic device and readable storage medium

Country Status (2)

Country Link
CN (1) CN114792285A (en)
WO (1) WO2023202570A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115797783A (en) * 2023-02-01 2023-03-14 北京有竹居网络技术有限公司 Method and device for generating barrier-free information, electronic equipment and storage medium
WO2023202570A1 (en) * 2022-04-21 2023-10-26 维沃移动通信有限公司 Image processing method and processing apparatus, electronic device and readable storage medium

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108805789B (en) * 2018-05-29 2022-06-03 厦门市美亚柏科信息股份有限公司 Method, device and equipment for removing watermark based on antagonistic neural network and readable medium
CN109035147B (en) * 2018-08-10 2023-12-12 Oppo广东移动通信有限公司 Image processing method and device, electronic device, storage medium and computer equipment
CN111260757A (en) * 2018-12-03 2020-06-09 马上消费金融股份有限公司 Image processing method and device and terminal equipment
CN110163194A (en) * 2019-05-08 2019-08-23 腾讯科技（深圳）有限公司 A kind of image processing method, device and storage medium
CN111062854B (en) * 2019-12-26 2023-08-25 Oppo广东移动通信有限公司 Method, device, terminal and storage medium for detecting watermark
CN113538273B (en) * 2021-07-13 2023-09-19 荣耀终端有限公司 Image processing method and image processing apparatus
CN114049280A (en) * 2021-11-25 2022-02-15 广州华多网络科技有限公司 Image erasing and repairing method and device, equipment, medium and product thereof
CN114792285A (en) * 2022-04-21 2022-07-26 维沃移动通信有限公司 Image processing method and processing device, electronic device and readable storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023202570A1 (en) * 2022-04-21 2023-10-26 维沃移动通信有限公司 Image processing method and processing apparatus, electronic device and readable storage medium
CN115797783A (en) * 2023-02-01 2023-03-14 北京有竹居网络技术有限公司 Method and device for generating barrier-free information, electronic equipment and storage medium

Also Published As

Publication number Publication date
WO2023202570A1 (en) 2023-10-26

Similar Documents

Publication Publication Date Title
CN114792285A (en) Image processing method and processing device, electronic device and readable storage medium
JP2020127194A (en) Computer system and program
CN112306347A (en) Image editing method, image editing device and electronic equipment
CN115599276A (en) Artificial intelligence AI-based three-dimensional model display method and system
CN115660969A (en) Image processing method, model training method, device, equipment and storage medium
CN115421631A (en) Interface display method and device
CN115410133A (en) Video dense prediction method and device
CN114049473A (en) Image processing method and device
CN114518821A (en) Application icon management method and device and electronic equipment
CN114529714A (en) Unmanned aerial vehicle detection method and device, computer equipment and storage medium
CN114758054A (en) Light spot adding method, device, equipment and storage medium
CN112258416A (en) Image processing method and device and electronic equipment
CN112465692A (en) Image processing method, device, equipment and storage medium
CN117496525A (en) Quality inspection method and device for characters in image, electronic equipment and readable storage medium
CN113923367B (en) Shooting method and shooting device
CN117593510A (en) Image processing method and electronic device
CN114500855A (en) Display method, display device, electronic apparatus, and readable storage medium
CN116975337A (en) Image searching method, device, electronic equipment and readable storage medium
CN116051366A (en) Image processing method, device, equipment and storage medium
CN117528179A (en) Video generation method and device
CN115423817A (en) Image segmentation method, device, electronic device and medium
CN115861579A (en) Display method and device thereof
CN115240260A (en) Image processing method and device
CN117331469A (en) Screen display method, device, electronic equipment and readable storage medium
CN114554098A (en) Display method, display device, electronic apparatus, and readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination