CN112215854A - Image processing method and device - Google Patents
Image processing method and device Download PDFInfo
- Publication number
- CN112215854A CN112215854A CN202011120091.4A CN202011120091A CN112215854A CN 112215854 A CN112215854 A CN 112215854A CN 202011120091 A CN202011120091 A CN 202011120091A CN 112215854 A CN112215854 A CN 112215854A
- Authority
- CN
- China
- Prior art keywords
- image
- processed
- images
- target image
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 51
- 238000013508 migration Methods 0.000 claims abstract description 124
- 230000005012 migration Effects 0.000 claims abstract description 124
- 238000012545 processing Methods 0.000 claims abstract description 101
- 238000000034 method Methods 0.000 claims description 31
- 230000011218 segmentation Effects 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 15
- 238000013527 convolutional neural network Methods 0.000 claims description 11
- 238000003860 storage Methods 0.000 claims description 9
- 230000000694 effects Effects 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 10
- 238000003709 image segmentation Methods 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 238000011176 pooling Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 244000025254 Cannabis sativa Species 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000001172 regenerating effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The application relates to an image processing method and device. The image processing method comprises the following steps: segmenting an original image to be processed to obtain at least two first images to be processed; processing each of the at least two first images to be processed by increasing pixel expansion area to obtain at least two second images to be processed; performing style migration processing on each of the at least two second images to be processed to obtain at least two target image units; and merging the at least two target image units based on a preset merging rule to obtain a target image. The image processing method provided by the application can achieve a better style migration effect on the image to be processed.
Description
Technical Field
The present application relates to the field of internet technologies, and in particular, to an image processing method and apparatus, a computing device, and a computer-readable storage medium.
Background
In the prior art, when a picture style is migrated, a certain formed style is often used as a template, and a certain picture is stylized through the template, so that the processed picture is output. This processing method of the prior art can be used for processing the picture with smaller size.
However, the sizes of the picture materials in the current games are often larger. The resolution commonly used in the industry is more than 1024 × 1024, and if the system is directly used, the system cannot normally operate due to the limitation of the performance of a machine graphics card; meanwhile, in the prior art, large-size images are segmented, the segmented images are merged after being subjected to style migration treatment, and a left gap exists at a merged boundary, so that a complete image cannot be formed, and the attractiveness is influenced.
Disclosure of Invention
In view of the above, the present application provides an image processing method and apparatus, a computing device and a computer-readable storage medium, so as to solve the technical defects in the prior art.
Specifically, the application provides the following technical scheme:
the application provides an image processing method, which comprises the following steps:
segmenting an original image to be processed to obtain at least two first images to be processed;
processing each of the at least two first images to be processed by increasing pixel expansion area to obtain at least two second images to be processed;
performing style migration processing on each of the at least two second images to be processed to obtain at least two target image units;
and merging the at least two target image units based on a preset merging rule to obtain a target image.
Optionally, for the image processing method, performing segmentation processing on an original image to be processed to obtain at least two first images to be processed includes:
and performing segmentation processing on the image to be processed based on a preset segmentation rule to obtain at least two first images to be processed, wherein each first image to be processed is a rectangular image.
Optionally, for the image processing method, performing pixel-added expansion region processing on each of at least two first images to be processed to obtain at least two second images to be processed includes:
expanding outwards a preset number of pixels based on the image edge of each first image to be processed in at least two first images to be processed, wherein the expanded pixels form the pixel expansion area;
and obtaining at least two second images to be processed based on the image content area corresponding to each first image to be processed and the pixel expansion area in at least two first images to be processed.
Optionally, the image processing method is characterized in that performing style migration processing on each of at least two second images to be processed to obtain at least two target image units, and includes:
acquiring a template image, and performing style migration processing on each second image to be processed of at least two second images to be processed based on the style of the template image to generate at least two target image units;
wherein each of the at least two target image units comprises an image content region after style migration and a pixel expansion region after style migration.
Optionally, for the image processing method, merging the at least two target image units based on a preset merging rule to obtain a target image, where the method includes:
arranging the at least two target image units;
merging the borders of every two adjacent target image units based on the image content area after the lattice migration in the target image units to obtain a pre-merged image;
and deleting the pixel expansion area after the grid migration in the pre-merged image to obtain a merged target image.
Optionally, for the image processing method, the arranging the at least two target image units includes:
and arranging the at least two target image units based on the arrangement sequence of the at least two first images to be processed corresponding to the image content areas after the lattice migration in the at least two target image units in the original images to be processed.
Optionally, with respect to the image processing method, when the image is a four-channel image including three primary color channels and an alpha channel, the method further includes:
and splitting the four-channel image to obtain an original three-primary-color channel, and generating an original image to be processed based on the original three-primary-color channel.
Optionally, for the image processing method, after obtaining the target image, the method further includes:
obtaining three primary color channels of the target image;
and generating an initial format image with four channels based on the four-channel image, and replacing the three-primary-color channels of the initial format image with the three-primary-color channels of the target image to obtain a final target image.
Optionally, for the image processing method, performing style migration processing on each of at least two second images to be processed based on the style of the template image, including:
and performing style migration processing on each second image to be processed of at least two second images to be processed based on the style of the template image by adopting a convolutional neural network.
The present application provides an image processing apparatus, the apparatus including:
the segmentation module is configured to segment an original image to be processed to obtain at least two first images to be processed;
the expansion module is configured to perform pixel-added expansion area processing on each of the at least two first images to be processed to obtain at least two second images to be processed;
the style migration module is configured to perform style migration processing on each of the at least two second images to be processed to obtain at least two target image units;
and the merging module is configured to merge the at least two target image units based on a preset merging rule to obtain a target image.
The present application provides a computing device comprising a memory, a processor and computer instructions stored on the memory and executable on the processor, wherein the processor implements the steps of the aforementioned image processing method when executing the instructions.
The present application provides a computer readable storage medium storing computer instructions, wherein the instructions, when executed by a processor, implement the steps of any of the aforementioned image processing methods.
The application provides an image processing method, which is characterized in that a large-size original image to be processed is firstly divided into at least two first images to be processed, and after the large-size image is divided into small-size images, style migration processing is respectively carried out, so that the requirements on the performance of a machine display card are reduced, and the operation is easier; then, each of the at least two first images to be processed is subjected to pixel expansion area increasing processing to obtain at least two second images to be processed, then, each of the at least two second images to be processed is subjected to style migration processing to obtain at least two target image units, the pixel expansion areas subjected to style migration are obtained after the style migration processing is carried out by increasing the pixel expansion areas, redundant data are generated at a combined frame, and gaps generated at the combined frame in the process of recombining the split images are filled; and finally, merging the at least two target image units based on a preset merging rule to obtain a target image, and further obtaining the target image with better style migration processing effect.
Drawings
Fig. 1 is a schematic flowchart of an image processing method according to an embodiment of the present application;
fig. 2a is a schematic diagram of image segmentation in an image processing method according to an embodiment of the present application;
fig. 2b is a schematic diagram of image segmentation in an image processing method according to an embodiment of the present application
Fig. 3 is a schematic structural diagram of adding a pixel expansion area in an image processing method according to an embodiment of the present application;
fig. 4 is a schematic flowchart of style migration processing in an image processing method according to an embodiment of the present application;
FIG. 5 is a schematic flowchart illustrating merging of target image units in an image processing method according to an embodiment of the present application;
fig. 6 is a schematic flowchart of an image processing method according to a second embodiment of the present application;
fig. 7 is a schematic flowchart of an example of image processing provided in the second embodiment of the present application;
fig. 8 is a schematic structural diagram of an image processing apparatus according to a third embodiment of the present application;
fig. 9 is a schematic structural diagram of a computing device according to a fourth embodiment of the present application.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.
The terminology used in the description of the one or more embodiments is for the purpose of describing the particular embodiments only and is not intended to be limiting of the description of the one or more embodiments. As used in one or more embodiments of the present specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used in one or more embodiments of the present specification refers to and encompasses any and all possible combinations of one or more of the associated listed items.
The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
First, the noun terms to which one or more embodiments of the present invention relate are explained.
Style migration: refers to converting the style of one picture into another picture. The style migration is to merge the contextual content of the original image with the style of the reference image, and the output image is close to the contextual content of the original image in content and close to the style of the reference image in style by the merging.
Pixel (pixel): is a basic unit of image display. In the whole image, the pixels can be regarded as a small grid which has a single color and cannot be subdivided into smaller elements or units, and the more pixels in a unit area represents the higher resolution, the clearer the displayed image is.
Template image: in this application, refers to images that are used as style templates in a style migration process.
A channel: refers to the components of an image. The different color channels register the status of a certain color.
Three primary color channels: refers to 3 color channels of red (R), green (G) and blue (B).
Alpha channel (Alpha channel, a channel): refers to the transparency and translucency of a picture. The Alpha value is generally between 0 and 1, wherein 0 is black and represents transparent; 1 is white, meaning opaque; the translucence is between 0 and 1.
Initial format image: referred to herein as a quad channel generated based picture, including the TGA format.
A convolutional neural network: is a deep neural network which is most effective in processing image tasks. Convolutional neural networks are feedforward neural networks composed of a number of network layers, each containing a number of computational units (neurons) for processing visual information. The computational units of each layer may be understood as a collection of picture filters, each layer being able to extract different specific features of the picture.
In the present application, an image processing method and apparatus, a computing device, and a computer-readable storage medium are provided, which are described in detail in the following embodiments one by one.
Example one
The present embodiment provides an image processing method, and referring to fig. 1, fig. 1 shows a flowchart of the image processing method provided by the present embodiment, which includes steps S101 to S104.
S101, carrying out segmentation processing on the original images to be processed to obtain at least two first images to be processed.
In the application, the original image to be processed is a static image, and suitable formats include JPEG and JPG formats.
Further, in this application, the segmenting the original image to be processed to obtain at least two first images to be processed includes:
and performing segmentation processing on the image to be processed based on a preset segmentation rule to obtain at least two first images to be processed, wherein each first image to be processed is a rectangular image.
Specifically, in the present application, the preset segmentation rule includes: equally dividing the image to be processed into n rectangular images with the same size; or the image to be processed is divided into n rectangular images with different sizes.
Specifically, as shown in fig. 2a, in which the resolution of the image to be processed is 1024 × 1024, the image to be processed is equally divided into 4 small-size partial resolution images a1 to a4 in fig. 2a, a1 to a4 are the first images to be processed, and the resolution of each of the first images to be processed is 512 × 512.
As shown in fig. 2b, since in the image to be processed (with a resolution of 1024 × 1024), the main content of the image is at the upper right of the image (for example, fig. 2b is a landscape photograph in which main scenes, such as trees, are mainly concentrated in the upper right region of the image, and the rest regions are scattered with grass, etc.), in order to ensure the integrity of the content information of the image as much as possible, the continuous part of the content information may be cut into the same region as much as possible during the process of dividing the original image to be processed, so in fig. 2b, the original image to be processed is not divided equally to obtain small-size resolution images b 1-b 4, and b 1-b 4 are the first image to be processed, where the resolution of the b2 region is 758 × 768.
Fig. 2a and fig. 2b are schematic schemes for performing original to-be-processed image segmentation in the present application, and may be specifically determined according to actual situations in a specific application process, for example, the original to-be-processed image segmentation is performed equally or not, and the original to-be-processed image segmentation is performed into 2 parts, 4 parts, or 8 parts, and the present application does not limit this.
S102, pixel expansion area increasing processing is carried out on each of the at least two first images to be processed to obtain at least two second images to be processed.
In the present application, the pixel extension region refers to a region in which pixels are added outside along an edge of an image based on an existing image, and the region formed by the added pixels is the pixel extension region.
Further, the processing of adding the pixel expansion area to each of the at least two first images to be processed to obtain at least two second images to be processed includes:
expanding outwards a preset number of pixels based on the image edge of each first image to be processed in at least two first images to be processed, wherein the expanded pixels form the pixel expansion area;
and obtaining at least two second images to be processed based on the image content area corresponding to each first image to be processed and the pixel expansion area in at least two first images to be processed.
Specifically, in the image processing method provided by the present application, an original image to be processed is segmented to obtain n first images to be processed, a preset number of pixels are added to each of the first images to be processed along an edge of the image, and all the added pixels are used as pixel extension areas. And because the newly added pixels do not contain image content and are only blank pixels, each first image to be processed is taken as an image content area (namely, a non-blank pixel area), and the extended pixel area corresponding to each first image to be processed and the image content area jointly form the second image to be processed.
As shown in fig. 3, fig. 3 shows a schematic structural diagram of a second to-be-processed image provided by the present application. Therein, two regions are included in fig. 3: a T region (diagonal line region) and a K region (dot region), wherein the T region represents an image content region (i.e., the first image to be processed), and the K region represents a pixel expansion region. The T area and the K area jointly form the second image to be processed.
Specifically, in fig. 3, the resolution of the first to-be-processed image in the T region is 256 × 256 (that is, the length of the image is 256 pixels, and the width of the image is 256 pixels), and in the present application, on the basis of the first to-be-processed image, 10 pixels are respectively extended outwards along four sides of the image, that is, the resolution of the obtained second to-be-processed image is 266 × 266 (that is, the length of the image is 266 pixels, and the width of the image is 266 pixels), where in the second to-be-processed image, the region (K region) formed by the added pixels is the pixel expansion region.
Further, in the image processing method provided by the present application, the number of the added pixels may be adjusted according to the result of the image processing, for example, 1, 10, 30, and the like, so as to achieve the best effect, which is not limited in the present application.
Further, since the original image to be processed is divided into a plurality of first images to be processed, the number of pixels added to each of the first images to be processed may be the same or different, and adjustment and optimization may be performed according to the image processing result.
S103, performing style migration processing on each of the at least two second images to be processed to obtain at least two target image units.
The style migration processing is as follows: the style of one image is transformed into the other image. For example, as shown in fig. 4, picture 1 is a real building image that needs style migration, picture 2 is a target style image, and then style migration is performed on picture 1 based on the style of picture 2 to generate picture 3, as can be seen from fig. 4, picture 3 fuses the style of patterns in picture 2 on the basis of the building of original picture 1, and a building image after style migration processing is obtained.
Further, in the image processing method provided by the present application, performing style migration processing on each of at least two second images to be processed to obtain at least two target image units includes:
acquiring a template image, and performing style migration processing on each second image to be processed of at least two second images to be processed based on the style of the template image to generate at least two target image units;
wherein each of the at least two target image units comprises an image content region after style migration and a pixel expansion region after style migration.
Specifically, in the image processing method provided by the application, a plurality of first images to be processed are generated on the basis of an original image to be processed, and then pixel extension areas are respectively added to each first image to be processed to obtain a plurality of second images to be processed; and then, a template image for performing style migration processing needs to be acquired, specifically, each second image to be processed is subjected to style migration processing respectively based on the style of the template image, in the process, not only is the image content area subjected to style migration processing, but also the pixel expansion area is subjected to style migration processing simultaneously, and finally, a plurality of target image units are acquired, and each target image unit has a corresponding image content area after style migration and a pixel expansion area after style migration.
Further, in the image processing method provided by the present application, performing style migration processing on each of at least two second to-be-processed images based on the style of the template image includes:
and performing style migration processing on each second image to be processed of at least two second images to be processed based on the style of the template image by adopting a convolutional neural network.
In particular, the convolutional neural network is one of the deep neural networks that is most efficient in processing the image task. Convolutional neural networks are feedforward neural networks composed of a number of network layers, each containing a number of computational units (neurons) for processing visual information. The computational units of each layer may be understood as a collection of picture filters, each layer being able to extract different specific features of the picture.
In the field of image style migration processing, a VGG (visual Geometry group) type convolutional neural network can be adopted for carrying out style migration, and the VGG type convolutional neural network comprises VGG-11, VGG-13, VGG-16, VGG-19 and the like.
The following takes VGG-19 as an example to specifically explain the training process of the image migration neural network model:
1) the image is read.
2) Feature extraction with VGG 19:
VGG19 may be divided into 5 blocks, each consisting of a number of convolutional layers followed by a pooling layer, the pooling layers of the 5 blocks included in VGG19 are all maximal pooling, except for the number of layers of convolutional layers, the first block has 2 convolutions (conv1_1 and conv1_2), the second block is also a 2-layer convolution, the last 3 blocks are all 4 convolutions, finally two fully connected layers (FC1 and FC2) and one softmax layer for classification. But the last two fully connected layers and the softmax layer are not needed in the style migration task.
Two pictures are: one of the pictures is used as content input (image required to perform style migration), and the other picture is used as style input (template image) and is respectively subjected to 5 blocks of VGG19 to obtain a feature map.
3) Model construction, loss value (loss):
defining a VGG19 model, wherein, in order to meet the input requirement of VGG19, the size of the input style picture and the size of the content picture need to be consistent.
4) Training with gradient descent.
Specifically, in the image processing method provided by the present application, the style migration processing performed on each second image to be processed is performed by using a trained style migration model (e.g., VGG 19). The style migration model already has a certain image style, and the style migration model can perform style migration based on the input image to generate a new image with a certain style.
In the application, the style migration processing is performed on each second image to be processed through the style migration model, and since the second image to be processed includes the image content area and the pixel extension area, in the processing process performed by using the style migration model, not only the image content area is subjected to style migration, but also the corresponding extension pixel area is subjected to style migration; and because the pixel extension area is added, in the process of carrying out style migration processing, the image content area can learn more style characteristics based on the template image, and the style migration effect is improved.
S103, merging the at least two target image units based on a preset merging rule to obtain a target image.
According to the foregoing, the style migration processing is performed on the plurality of second images to be processed to obtain a plurality of target image units; and then merging the target image units based on a preset merging rule to obtain a target image. The target image is an image after the style migration corresponding to the original image to be processed, and the resolution of the target image is the same as the size of the original image to be processed.
Specifically, in the image processing method provided in the present application, merging the at least two target image units based on a preset merging rule to obtain a target image includes:
arranging the at least two target image units;
merging the borders of every two adjacent target image units based on the image content area after the lattice migration in the target image units to obtain a pre-merged image;
and deleting the pixel expansion area after the grid migration in the pre-merged image to obtain a merged target image.
Specifically, referring to fig. 5, fig. 5 is a schematic diagram illustrating merging of two adjacent target image units. Wherein, A1 and A2 are two adjacent target image units obtained by style migration processing, wherein, in A1 and A2, the middle grid part represents the image content area after style migration; the external blank frame represents an extended pixel area after the style migration; where a and b are the borders of the style-migrated image content areas in a1, a2, respectively. In merging, merging is performed along the borders of the adjacent target image units based on the region of the image content after the lattice migration in the target image units, i.e. merging is performed along the borders a and b, as shown in fig. 5, at the merged borders, the pixel extension regions after the style migration in a1 and a2 overlap, so that redundant data is generated at the merged borders based on the overlapped pixel extension regions after the style migration, resulting in the pre-merged image.
The redundant data refers to: refers to the duplication between data, and can also be said to be the phenomenon that the same data is stored in different data files.
Further, deleting the pixel extension area with style migration in the pre-merged image to obtain a target image. Specifically, in the process, since redundant data exists at the merged frame (a, b), after the extended pixels of the style migration are cut off, the redundant data fills the "gap" existing at the merged frame, and finally the target image without the merged gap is obtained, so that the target image with better style migration processing effect is obtained.
Further, arranging the at least two target image units includes:
and arranging the at least two target image units based on the arrangement sequence of the at least two first images to be processed corresponding to the image content areas after the lattice migration in the at least two target image units in the original images to be processed.
Specifically, for example, in fig. 2a, the original image to be processed is divided into a1-a4 four first images to be processed, and four target image units a1 '-a 4' are generated based on a1-a4, then in the process of merging, the a1 '-a 4' is arranged according to the position order of the corresponding a1-a4 in the original image to be processed.
The application provides an image processing method through the steps S101-S104, wherein a large-size original image to be processed is firstly divided into at least two first images to be processed, and the large-size image is divided into small-size images and then subjected to style migration processing, so that the requirement on the performance of a machine display card is lowered, and the operation is easier; then, each of the at least two first images to be processed is subjected to pixel expansion area increasing processing to obtain at least two second images to be processed, then, each of the at least two second images to be processed is subjected to style migration processing to obtain at least two target image units, the pixel expansion areas subjected to style migration are obtained after the style migration processing is carried out by increasing the pixel expansion areas, redundant data are generated at a combined frame, and gaps generated at the combined frame in the process of recombining the split images are filled; and finally, merging the at least two target image units based on a preset merging rule to obtain a target image, and further obtaining the target image with better style migration processing effect.
Example two
The present embodiment provides an image processing method, and referring to fig. 6, fig. 6 shows a flowchart of the image processing method provided by the present embodiment, which includes steps S601 to S607.
S601, when the image is a four-channel image comprising three primary color channels and an alpha channel, splitting the four-channel image to obtain an original three primary color channel, and generating an original image to be processed based on the original three primary color channel.
Specifically, the format of the original image to be processed adopted in the image processing method provided by the application requires a picture of a three primary color channel.
Wherein the three primary color channels: refers to 3 color channels of red (R), green (G) and blue (B).
Alpha channel (Alpha channel, a channel for short): refers to the transparency and translucency of a picture. The Alpha value is generally between 0 and 1, wherein 0 is black and represents transparent; 1 is white, meaning opaque; the translucence is between 0 and 1.
Since the game picture usually has an Alpha channel, the style migration processing cannot be performed by using a four-channel picture with an Alpha channel.
Specifically, in this case, the image processing method provided by the present application includes the steps of:
firstly, splitting an original four-channel image to obtain an original three-primary-color channel, namely an RGB three-primary-color channel;
and then generating an original image to be processed in a target format based on the split original RGB three primary color channel, wherein the target format comprises JPEG and JPG formats.
S602, carrying out segmentation processing on the original images to be processed to obtain at least two first images to be processed.
Specifically, the segmenting the original image to be processed to obtain at least two first images to be processed includes:
and performing segmentation processing on the image to be processed based on a preset segmentation rule to obtain at least two first images to be processed, wherein each first image to be processed is a rectangular image.
S603, carrying out pixel expansion area increasing processing on each of the at least two first images to be processed to obtain at least two second images to be processed.
Specifically, the processing of increasing the pixel expansion area for each of the at least two first images to be processed to obtain at least two second images to be processed includes:
expanding outwards a preset number of pixels based on the image edge of each first image to be processed in at least two first images to be processed, wherein the expanded pixels form the pixel expansion area;
and obtaining at least two second images to be processed based on the image content area corresponding to each first image to be processed and the pixel expansion area in at least two first images to be processed.
S604, performing style migration processing on each of the at least two second images to be processed to obtain at least two target image units.
Specifically, performing style migration processing on each of at least two second images to be processed to obtain at least two target image units includes:
acquiring a template image, and performing style migration processing on each second image to be processed of at least two second images to be processed based on the style of the template image to generate at least two target image units;
wherein each of the at least two target image units comprises an image content region after style migration and a pixel expansion region after style migration.
Further, performing style migration processing on each second image to be processed of at least two second images to be processed based on the style of the template image, including:
and performing style migration processing on each second image to be processed of at least two second images to be processed based on the style of the template image by adopting a convolutional neural network.
And S605, merging the at least two target image units based on a preset merging rule to obtain a target image.
Specifically, merging the at least two target image units based on a preset merging rule to obtain a target image, including:
arranging the at least two target image units;
merging the borders of every two adjacent target image units based on the image content area after the lattice migration in the target image units to obtain a pre-merged image;
and deleting the pixel expansion area after the grid migration in the pre-merged image to obtain a merged target image.
Further, arranging the at least two target image units includes:
and arranging the at least two target image units based on the arrangement sequence of the at least two first images to be processed corresponding to the image content areas after the lattice migration in the at least two target image units in the original images to be processed.
The specific process of generating the target image based on the original image to be processed in steps S602 to S605 is described in detail in the foregoing embodiment, and therefore is not described herein again.
And S606, obtaining three primary color channels of the target image.
Specifically, the three primary color channel is a three primary color channel of the target image generated after the style migration processing, and is different from the original RGB three primary color channel.
S607, generating an initial format image with four channels based on the four-channel image, and replacing the three primary color channels of the initial format image with the three primary color channels of the target image to obtain a final target image.
Specifically, the initial format image generated based on the four-channel image includes: TGA format.
And then replacing the three-primary color channel of the initial format image with the three-primary color channel of the obtained target image to obtain a final target image corresponding to the style migration processing of the original four-channel image.
Further, by way of example, referring to fig. 7, fig. 7 shows a flow chart of the style migration process performed on the four-channel image.
Firstly, splitting an original RGBA picture (four-channel picture) to obtain RGB three channels, and generating a JPG format picture based on the RGB three channels obtained by splitting, wherein the JPG format picture is the original image to be processed; converting the original RGBA picture into a TGA format picture;
then, carrying out image cutting, pixel expansion area increasing and style migration processing on the JPG format picture, and finally merging to obtain a JPG' format picture after the style migration processing;
obtaining RGB three channels based on the obtained JPG format picture;
and finally, replacing the RGB three channels in the TGA format picture with the RGB three channels to obtain the TGA format picture after style migration, namely the final target image.
The method comprises the steps of splitting a four-channel image into three primary color channels, regenerating a three-channel image which accords with an image processing format, generating a target image with a style migration based on the three-channel image, replacing the three primary color channels corresponding to the target image with the three primary color channels in the original four-channel image to obtain a final target image, and performing style migration processing on the four-channel image through the steps; meanwhile, the defects that the original image is too large in size and gaps are generated when the original image is cut and then combined are overcome, and the image style migration processing effect is improved.
EXAMPLE III
The embodiment provides an image processing apparatus, referring to fig. 8, fig. 8 shows a structural diagram of the image processing apparatus provided by the present application, including the following modules:
a segmentation module 810 configured to perform segmentation processing on an original image to be processed to obtain at least two first images to be processed;
an expansion module 820 configured to perform pixel-added expansion area processing on each of at least two first images to be processed to obtain at least two second images to be processed;
the style migration module 830 is configured to perform style migration processing on each of the at least two second images to be processed to obtain at least two target image units;
a merging module 840 configured to merge the at least two target image units based on a preset merging rule to obtain a target image.
In particular, the segmentation module 810 is further configured to: and performing segmentation processing on the image to be processed based on a preset segmentation rule to obtain at least two first images to be processed, wherein each first image to be processed is a rectangular image.
Specifically, the extension module 820 is further configured to:
expanding outwards a preset number of pixels based on the image edge of each first image to be processed in at least two first images to be processed, wherein the expanded pixels form the pixel expansion area;
and obtaining at least two second images to be processed based on the image content area corresponding to each first image to be processed and the pixel expansion area in at least two first images to be processed.
Specifically, the style migration module 830 is further configured to:
acquiring a template image, and performing style migration processing on each second image to be processed of at least two second images to be processed based on the style of the template image to generate at least two target image units;
wherein each of the at least two target image units comprises an image content region after style migration and a pixel expansion region after style migration.
Specifically, the style migration module 830 is further configured to:
and performing style migration processing on each second image to be processed of at least two second images to be processed based on the style of the template image by adopting a convolutional neural network.
Specifically, the merging module 840 is further configured to:
arranging the at least two target image units;
merging the borders of every two adjacent target image units based on the image content area after the lattice migration in the target image units to obtain a pre-merged image;
and deleting the pixel expansion area after the grid migration in the pre-merged image to obtain a merged target image.
Specifically, the merging module 840 is further configured to:
and arranging the at least two target image units based on the arrangement sequence of the at least two first images to be processed corresponding to the image content areas after the lattice migration in the at least two target image units in the original images to be processed.
Specifically, the image processing apparatus provided by the present application further comprises a preprocessing module configured to:
when the image is a four-channel image comprising three primary color channels and an alpha channel, splitting the four-channel image to obtain an original three primary color channel, and generating an original image to be processed based on the original three primary color channel.
Specifically, the image processing apparatus provided by the present application further comprises a post-processing module configured to:
obtaining three primary color channels of the target image;
and generating an initial format image with four channels based on the four-channel image, and replacing the three-primary-color channels of the initial format image with the three-primary-color channels of the target image to obtain a final target image.
The application provides an image processing device, which can divide a large-size image into small-size images to respectively perform style migration processing, reduces the requirements on the performance of a machine display card and is easier to operate; and by adding the pixel extension area, the pixel extension area of the style migration is obtained after the style migration processing is carried out, redundant data is generated at the combined frame, gaps generated at the combined frame in the process of recombining the split images are filled, and the target image with better style migration processing effect is obtained.
The above is a schematic configuration of an image processing apparatus of the present embodiment. It should be noted that the technical solution of the apparatus belongs to the same concept as the technical solution of the method of the image processing apparatus described above, and for details that are not described in detail in the technical solution of the image processing apparatus, reference may be made to the description of the technical solution of the image processing method described above. The specific content included in the image processing method has been provided in the foregoing embodiments, and is not described herein again.
Example four
The present embodiment provides a computing device 900, as shown in FIG. 9.
FIG. 9 is a block diagram that illustrates a computing device 900 in accordance with one embodiment of the present description. Components of the computing device 900 include, but are not limited to, a memory 910 and a processor 920. The processor 920 is coupled to the memory 910 via a bus 930, and a database 950 is used to store data.
Computing device 900 also includes access device 940, access device 940 enabling computing device 900 to communicate via one or more networks 960. Examples of such networks include the Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the internet. Access device 940 may include one or more of any type of network interface (e.g., a Network Interface Card (NIC)) whether wired or wireless, such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a bluetooth interface, a Near Field Communication (NFC) interface, and so forth.
In one embodiment of the present description, the above-described components of computing device 900, as well as other components not shown in FIG. 9, may also be connected to each other, such as by a bus. It should be understood that the block diagram of the computing device architecture shown in FIG. 9 is for purposes of example only and is not limiting as to the scope of the description. Those skilled in the art may add or replace other components as desired.
Computing device 900 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., tablet, personal digital assistant, laptop, notebook, netbook, etc.), a mobile phone (e.g., smartphone), a wearable computing device (e.g., smartwatch, smartglasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. Computing device 900 may also be a mobile or stationary server.
Wherein, the processor 920 may execute the steps in the image processing method provided by the foregoing embodiment. The specific steps are not described in detail in this embodiment.
An embodiment of the present application further provides a computer readable storage medium, which stores computer instructions, and the instructions, when executed by a processor, implement the steps in the image processing method as described above.
The above is an illustrative scheme of a computer-readable storage medium of the present embodiment. It should be noted that the technical solution of the storage medium belongs to the same concept as the technical solution of the image processing method, and details that are not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the image processing method.
The computer instructions comprise computer program code which may be in the form of source code, object code, an executable file or some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
It should be noted that, for the sake of simplicity, the above-mentioned method embodiments are described as a series of acts or combinations, but those skilled in the art should understand that the present application is not limited by the described order of acts, as some steps may be performed in other orders or simultaneously according to the present application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
The preferred embodiments of the present application disclosed above are intended only to aid in the explanation of the application. Alternative embodiments are not exhaustive and do not limit the invention to the precise embodiments described. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the application and the practical application, to thereby enable others skilled in the art to best understand and utilize the application. The application is limited only by the claims and their full scope and equivalents.
Claims (12)
1. An image processing method, comprising:
segmenting an original image to be processed to obtain at least two first images to be processed;
processing each of the at least two first images to be processed by increasing pixel expansion area to obtain at least two second images to be processed;
performing style migration processing on each of the at least two second images to be processed to obtain at least two target image units;
and merging the at least two target image units based on a preset merging rule to obtain a target image.
2. The method according to claim 1, wherein the segmenting the original image to be processed to obtain at least two first images to be processed comprises:
and performing segmentation processing on the image to be processed based on a preset segmentation rule to obtain at least two first images to be processed, wherein each first image to be processed is a rectangular image.
3. The method according to claim 1, wherein performing the area-added pixel expansion process on each of the at least two first to-be-processed images to obtain at least two second to-be-processed images comprises:
expanding outwards a preset number of pixels based on the image edge of each first image to be processed in at least two first images to be processed, wherein the expanded pixels form the pixel expansion area;
and obtaining at least two second images to be processed based on the image content area corresponding to each first image to be processed and the pixel expansion area in at least two first images to be processed.
4. The method according to claim 1, wherein performing style migration processing on each of the at least two second images to be processed to obtain at least two target image units comprises:
acquiring a template image, and performing style migration processing on each second image to be processed of at least two second images to be processed based on the style of the template image to generate at least two target image units;
wherein each of the at least two target image units comprises an image content region after style migration and a pixel expansion region after style migration.
5. The method according to claim 4, wherein merging the at least two target image units based on a preset merging rule to obtain a target image comprises:
arranging the at least two target image units;
merging the borders of every two adjacent target image units based on the image content area after the lattice migration in the target image units to obtain a pre-merged image;
and deleting the pixel expansion area after the grid migration in the pre-merged image to obtain a merged target image.
6. The method of claim 5, wherein arranging the at least two target-image cells comprises:
and arranging the at least two target image units based on the arrangement sequence of the at least two first images to be processed corresponding to the image content areas after the lattice migration in the at least two target image units in the original images to be processed.
7. The method of claim 1, wherein when the image is a four-channel image including three primary color channels and an alpha channel, the method further comprises:
and splitting the four-channel image to obtain an original three-primary-color channel, and generating an original image to be processed based on the original three-primary-color channel.
8. The method of claim 7, wherein after obtaining the target image, the method further comprises:
obtaining three primary color channels of the target image;
and generating an initial format image with four channels based on the four-channel image, and replacing the three-primary-color channels of the initial format image with the three-primary-color channels of the target image to obtain a final target image.
9. The method according to claim 4, wherein performing style migration processing on each of at least two second images to be processed based on the style of the template image comprises:
and performing style migration processing on each second image to be processed of at least two second images to be processed based on the style of the template image by adopting a convolutional neural network.
10. An image processing apparatus, characterized in that the apparatus comprises:
the segmentation module is configured to segment an original image to be processed to obtain at least two first images to be processed;
the expansion module is configured to perform pixel-added expansion area processing on each of the at least two first images to be processed to obtain at least two second images to be processed;
the style migration module is configured to perform style migration processing on each of the at least two second images to be processed to obtain at least two target image units;
and the merging module is configured to merge the at least two target image units based on a preset merging rule to obtain a target image.
11. A computing device comprising a memory, a processor and computer instructions stored on the memory and executable on the processor, wherein the steps of the image processing method of any of claims 1 to 9 are implemented when the processor executes the instructions.
12. A computer-readable storage medium storing computer instructions, which when executed by a processor, perform the steps of the image processing method of any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011120091.4A CN112215854A (en) | 2020-10-19 | 2020-10-19 | Image processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011120091.4A CN112215854A (en) | 2020-10-19 | 2020-10-19 | Image processing method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112215854A true CN112215854A (en) | 2021-01-12 |
Family
ID=74055896
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011120091.4A Pending CN112215854A (en) | 2020-10-19 | 2020-10-19 | Image processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112215854A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113763233A (en) * | 2021-08-04 | 2021-12-07 | 深圳盈天下视觉科技有限公司 | Image processing method, server and photographing device |
| WO2022199364A1 (en) * | 2021-03-24 | 2022-09-29 | 北京字节跳动网络技术有限公司 | Image processing method and apparatus, and storage medium and device |
| CN115272146A (en) * | 2022-07-27 | 2022-11-01 | 天翼爱音乐文化科技有限公司 | Stylized image generation method, system, device and medium |
| WO2023272432A1 (en) * | 2021-06-28 | 2023-01-05 | 华为技术有限公司 | Image processing method and image processing apparatus |
| WO2024104477A1 (en) * | 2022-11-18 | 2024-05-23 | 北京字跳网络技术有限公司 | Image generation method and apparatus, electronic device, and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109859211A (en) * | 2018-12-28 | 2019-06-07 | 努比亚技术有限公司 | A kind of image processing method, mobile terminal and computer readable storage medium |
| CN111415299A (en) * | 2020-03-26 | 2020-07-14 | 浙江科技学院 | High-resolution image style migration method |
| CN111445387A (en) * | 2020-06-16 | 2020-07-24 | 浙江科技学院 | High-resolution image style migration method based on random rearrangement of image blocks |
| CN111652830A (en) * | 2020-06-28 | 2020-09-11 | Oppo广东移动通信有限公司 | Image processing method and device, computer readable medium and terminal equipment |
-
2020
- 2020-10-19 CN CN202011120091.4A patent/CN112215854A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109859211A (en) * | 2018-12-28 | 2019-06-07 | 努比亚技术有限公司 | A kind of image processing method, mobile terminal and computer readable storage medium |
| CN111415299A (en) * | 2020-03-26 | 2020-07-14 | 浙江科技学院 | High-resolution image style migration method |
| CN111445387A (en) * | 2020-06-16 | 2020-07-24 | 浙江科技学院 | High-resolution image style migration method based on random rearrangement of image blocks |
| CN111652830A (en) * | 2020-06-28 | 2020-09-11 | Oppo广东移动通信有限公司 | Image processing method and device, computer readable medium and terminal equipment |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022199364A1 (en) * | 2021-03-24 | 2022-09-29 | 北京字节跳动网络技术有限公司 | Image processing method and apparatus, and storage medium and device |
| WO2023272432A1 (en) * | 2021-06-28 | 2023-01-05 | 华为技术有限公司 | Image processing method and image processing apparatus |
| CN113763233A (en) * | 2021-08-04 | 2021-12-07 | 深圳盈天下视觉科技有限公司 | Image processing method, server and photographing device |
| CN115272146A (en) * | 2022-07-27 | 2022-11-01 | 天翼爱音乐文化科技有限公司 | Stylized image generation method, system, device and medium |
| WO2024104477A1 (en) * | 2022-11-18 | 2024-05-23 | 北京字跳网络技术有限公司 | Image generation method and apparatus, electronic device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112215854A (en) | Image processing method and device | |
| US20200202601A1 (en) | Predicting Patch Displacement Maps Using A Neural Network | |
| JP2023539691A (en) | Human image restoration methods, devices, electronic devices, storage media, and program products | |
| JP4142614B2 (en) | Trapping method, trapping program, trapping apparatus, and printing system | |
| CN112733044B (en) | Recommended image processing method, apparatus, device and computer-readable storage medium | |
| CN111783735B (en) | Steel document analytic system based on artificial intelligence | |
| CN111353956B (en) | Image restoration method and device, computer equipment and storage medium | |
| CA3154893A1 (en) | Image color transferring method, device, computer equipment and storage medium | |
| CN111681165A (en) | Image processing method, image processing device, computer equipment and computer readable storage medium | |
| CN113409411A (en) | Rendering method and device of graphical interface, electronic equipment and storage medium | |
| JP5526874B2 (en) | Image processing apparatus and image processing program | |
| CN113963081A (en) | Intelligent image chart synthesis method based on graph convolution network | |
| CN115830592A (en) | Overlapping cervical cell segmentation method and system | |
| CN115270184A (en) | Video desensitization method, vehicle video desensitization method and vehicle-mounted processing system | |
| CN112733861A (en) | Text erasing and character matting method based on U-shaped residual error network | |
| CN107533760A (en) | A kind of image partition method and device | |
| CN117058554A (en) | Power equipment target detection method, model training method and device | |
| CN114863435A (en) | Text extraction method and device | |
| CN116310666A (en) | Remote sensing image self-supervision training method and device and electronic equipment | |
| CN114022458A (en) | Skeleton detection method and device, electronic equipment and computer readable storage medium | |
| JP4024744B2 (en) | Trapping method, trapping apparatus, trapping program, and printing system | |
| CN114529624A (en) | Image color matching method and system and image generation method and system | |
| JP4654237B2 (en) | Dropout color processing device | |
| CN117351032B (en) | Seal removing method and system | |
| JP2024046749A (en) | Method and apparatus for transforming an input image based on target style and target color information - Patents.com |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 519000 Room 102, 202, 302 and 402, No. 325, Qiandao Ring Road, Tangjiawan Town, high tech Zone, Zhuhai City, Guangdong Province, Room 102 and 202, No. 327 and Room 302, No. 329 Applicant after: Zhuhai Jinshan Digital Network Technology Co.,Ltd. Address before: 519000 Room 102, 202, 302 and 402, No. 325, Qiandao Ring Road, Tangjiawan Town, high tech Zone, Zhuhai City, Guangdong Province, Room 102 and 202, No. 327 and Room 302, No. 329 Applicant before: ZHUHAI KINGSOFT ONLINE GAME TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information |