CN113628105A

CN113628105A - Image processing method, device, storage medium and processor

Info

Publication number: CN113628105A
Application number: CN202010379128.9A
Authority: CN
Inventors: 侯文迪; 吴腾飞; 曹文冬; 刘尚; 崔苗苗
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-05-07
Filing date: 2020-05-07
Publication date: 2021-11-09

Abstract

The invention discloses an image processing method, an image processing device, a storage medium and a processor. Wherein, the method comprises the following steps: acquiring an original image of a media file; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and adjusting the original image based on the second element information to obtain a target image of the media file. The invention solves the technical problem that the image of the media file is single.

Description

Image processing method, device, storage medium and processor

Technical Field

The present invention relates to the field of image processing, and in particular, to an image processing method, an image processing apparatus, a storage medium, and a processor.

Background

At present, when an image of a media file is generated, data layout information in the media file is generally consistent, and the repetition degree of element information of the media file is also high, so that the diversity of the generated image of the media file is insufficient, and the technical problem that the image of the media file is relatively single exists.

In view of the above-mentioned technical problem of single image of the media file, no effective solution has been proposed at present.

Disclosure of Invention

The embodiment of the invention provides an image processing method, an image processing device, a storage medium and a processor, which are used for at least solving the technical problem that the image of a media file is relatively single.

According to an aspect of an embodiment of the present invention, there is provided an image processing method. The method comprises the following steps: acquiring an original image of a media file; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and adjusting the original image based on the second element information to obtain a target image of the media file.

According to another aspect of the embodiments of the present invention, there is also provided another image processing method. The method comprises the following steps: displaying an original image of the media file on an operation interface; reading first element information of an original image, and acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and displaying a target image of the media file obtained by adjusting the original image based on the second element information in the browsing area of the operation interface.

According to another aspect of the embodiments of the present invention, there is also provided another image processing method. The method comprises the following steps: acquiring an original image of a media file, wherein the original image comprises an item to be sold; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value and is used for modifying the article; and adjusting the original image based on the second element information to obtain a target image of the media file.

According to another aspect of the embodiments of the present invention, there is also provided another image processing method. The method comprises the following steps: displaying an original image of the media file on an operation interface; responding to an input operation instruction acting on the operation interface, and acquiring second element information; reading first element information of an original image corresponding to second element information, wherein the similarity between the second element information and the first element information is larger than a target threshold value; and displaying a target image of the media file obtained by adjusting the original image based on the second element information in the browsing area of the operation interface.

According to another aspect of the embodiments of the present invention, there is provided an image processing apparatus. The apparatus may include: a first acquisition unit for acquiring an original image of a media file; a second acquisition unit configured to acquire first element information of an original image; a third acquiring unit configured to acquire second element information corresponding to the first element information, wherein a similarity between the second element information and the first element information is greater than a target threshold; and the first adjusting unit is used for adjusting the original image based on the second element information to obtain a target image of the media file.

According to another aspect of the embodiments of the present invention, there is also provided another image processing apparatus. The apparatus may include: the first display unit is used for displaying an original image of the media file on the operation interface; the reading unit is used for reading first element information of an original image and acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and the second display unit is used for displaying a target image of the media file obtained by adjusting the original image based on the second element information in the browsing area of the operation interface.

According to another aspect of the embodiments of the present invention, there is also provided another image processing apparatus. The apparatus may include: the fourth acquisition unit is used for acquiring an original image of the media file, wherein the original image comprises an article to be sold; a fifth acquiring unit configured to acquire first element information of an original image; a sixth obtaining unit, configured to obtain second element information corresponding to the first element information, where a similarity between the second element information and the first element information is greater than a target threshold, and the second obtaining unit is configured to modify the article; and the second adjusting unit is used for adjusting the original image based on the second element information to obtain a target image of the media file.

According to another aspect of embodiments of the present invention, there is provided a storage medium. The storage medium includes a stored program, wherein the program, when executed, controls an apparatus in which the storage medium is located to perform the steps of: acquiring an original image of a media file; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and adjusting the original image based on the second element information to obtain a target image of the media file.

According to another aspect of embodiments of the present invention, a processor is provided. The processor is used for running the program, wherein the program executes the following steps: acquiring an original image of a media file; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and adjusting the original image based on the second element information to obtain a target image of the media file.

According to another aspect of the embodiments of the present invention, a mobile terminal is provided. The mobile terminal includes: a processor; the image acquisition device is used for acquiring an original image of the media file; and the memory is connected with the image acquisition device and used for providing instructions for the processor to process the following processing steps: acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and adjusting the original image based on the second element information to obtain a target image of the media file.

In the embodiment of the invention, an original image of a media file is obtained; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and adjusting the original image based on the second element information to obtain a target image of the media file. That is to say, the original image of the media file based on the internet of the application obtains the first element information from the original image, and then selects the appropriate second element information based on the first element information to adjust the original image, so that the purpose of generating the target image from the original image is achieved, the problem that the generated image is not diverse enough due to the fact that the data layout information of the media file is consistent and the repetition degree of the element information is high is avoided, the technical problem that the image of the media file is single is solved, and the technical effect of improving the diversity of the image of the media file is achieved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

fig. 1 is a block diagram of a hardware configuration of a computer terminal (or mobile device) for implementing an image processing method according to an embodiment of the present invention;

FIG. 2 is a flow chart of a method of image processing according to an embodiment of the present invention;

FIG. 3 is a flow diagram of another image processing method according to an embodiment of the invention;

FIG. 4 is a flow diagram of another image processing method according to an embodiment of the invention;

FIG. 5 is a flow chart of another method of generating an image of a media file according to an embodiment of the present invention;

FIG. 6A is a schematic illustration of an original image of a banner in accordance with an embodiment of the present invention;

FIG. 6B is a schematic diagram of a construction matrix according to an embodiment of the invention;

FIG. 7A is a schematic diagram of adding a dotted decoration element on the basis of a structural matrix according to an embodiment of the present invention;

FIG. 7B is a schematic diagram of a first result image obtained by adding a decoration element to a structural matrix according to an embodiment of the present invention;

FIG. 8A is a schematic diagram of adding a body modifier element to a structural matrix according to an embodiment of the present invention;

FIG. 8B is a diagram of a second resulting image obtained by adding a body-modifying element to a structural matrix according to an embodiment of the present invention;

FIG. 9 is a schematic illustration of an interactive interface of an image processing method according to an embodiment of the present invention;

FIG. 10 is a diagram illustrating advertisement image generation based on image parsing in an e-market scene according to an embodiment of the invention;

fig. 11 is a schematic diagram of an image processing apparatus according to an embodiment of the present invention;

fig. 12 is a schematic diagram of another image processing apparatus according to an embodiment of the present invention;

fig. 13 is a schematic diagram of another image processing apparatus according to an embodiment of the present invention; and

fig. 14 is a block diagram of a mobile terminal according to an embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

First, some terms or terms appearing in the description of the embodiments of the present application are applicable to the following explanations:

banner advertisement (banner) of website page mainly embodies central idea, vividly expresses most main emotional thought or propaganda center;

the layout is consistent, and the structural matrix corresponding to the element positions is designed to embody the layout;

visual styles including visual underlying features such as color, texture, etc.;

a Graph convolution Network model (GCN) is convolution-able, and filter parameters can be shared at all positions or one local position in the Graph;

the depth information, which is used to indicate the distance, may be represented by a depth map, the pixels of which are 0-256, wherein the closer to the acquisition device, the smaller the pixel value, i.e. the dark color, and the farther from the acquisition device, the larger the pixel value, i.e. the light color.

Example 1

There is also provided, in accordance with an embodiment of the present invention, an embodiment of an image processing method, to note that the steps illustrated in the flowchart of the figure may be performed in a computer system such as a set of computer-executable instructions, and that while a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order different than here.

The method provided by the first embodiment of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Fig. 1 is a block diagram of a hardware configuration of a computer terminal (or mobile device) for implementing an image processing method according to an embodiment of the present invention. As shown in fig. 1, the computer terminal 10 (or mobile device 10) may include one or more (shown as 102a, 102b, … …, 102 n) processors 102 (the processors 102 may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. Besides, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of the I/O interface), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.

It should be noted that the one or more processors 102 and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuit may be a single stand-alone processing module, or incorporated in whole or in part into any of the other elements in the computer terminal 10 (or mobile device). As referred to in the embodiments of the application, the data processing circuit acts as a processor control (e.g. selection of a variable resistance termination path connected to the interface).

The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the image processing method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, so as to implement the image processing method described above. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device 106 is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.

The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the computer terminal 10 (or mobile device).

It should be noted here that in some alternative embodiments, the computer device (or mobile device) shown in fig. 1 described above may include hardware elements (including circuitry), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. It should be noted that fig. 1 is only one example of a particular specific example and is intended to illustrate the types of components that may be present in the computer device (or mobile device) described above.

In the operating environment shown in fig. 1, the present application provides a monitoring method as shown in fig. 2. It should be noted that the method for generating animation of this embodiment may be executed by the mobile terminal of the embodiment shown in fig. 1.

Fig. 2 is a flowchart of an image processing method according to an embodiment of the present invention. As shown in fig. 2, the method may include the steps of:

step S202, an original image of the media file is acquired.

In the technical solution provided in step S202 of the present invention, the media file is a media file for which a new image needs to be expanded, and may be a banner advertisement (banner) of a website page, which may mainly embody a central idea and vividly express a most main emotional thought or a propaganda center. Optionally, the media file may be a media file in an e-market scene, for example, a shopping advertisement in the e-market scene.

The method includes the steps of obtaining an original image of a media file, wherein the original image is the most initial image of the media file, for example, an original banner image, and can be used as a template image of an image to be developed, and the original image content of the media file is included. Alternatively, the original image of this embodiment may use an image file in the GIF format, may use a static graphic, and may be stitched into an animated image by using a plurality of frame images.

In step S204, first element information of the original image is acquired.

In the technical solution provided by step S204 of the present invention, after the original image of the media file is acquired, the first element information of the original image is acquired.

In this embodiment, the first element information is information of a design constituent element of the original image, and may include information of a position, a size, a type, and the like of the design constituent element. Wherein, the type can include but is not limited to a commodity type, a case type, a decoration type, an identification (logo) type, a background type, etc.

Optionally, the first element information of this embodiment includes element information that needs to be replaced, and may also include element information that needs to be decorated.

In step S206, second element information corresponding to the first element information is acquired.

In the technical solution provided in step S206 of the present invention, after the first element information of the original image is acquired, second element information corresponding to the first element information is acquired, wherein a similarity between the second element information and the first element information is greater than a target threshold.

In this embodiment, the second element information has a corresponding relationship with the first element information, and the second element information may be acquired in the target database based on the first element information. The target database may be a database including a large amount of design data of the media file, a similarity between the second element information and the first element information is greater than a target threshold, and a similarity between a feature of the second element information and a feature of the first element information is greater than a target threshold, that is, the feature of the second element information is similar to the feature of the first element information, where the feature may be any one or more of a color feature, a depth information feature, a style feature, and the like, so that the second element information is element information suitable for replacing or modifying the first element information, so that a target image of the media file obtained by adjusting the original image based on the second element information has a greater similarity with the original image.

And step S208, adjusting the original image based on the second element information to obtain a target image of the media file.

In the technical solution provided in step S208 of the present invention, after the second element information corresponding to the first element information is obtained, the original image may be adjusted based on the second element information, that is, on the basis of the original image, the target image of the media file is expanded through the second element information, and the visual similarity between the target image and the original image is smaller than a certain threshold. Optionally, the target image is at least one image visually similar to the original image, which may be referred to as a result image of adjusting the original image, so as to achieve the purpose of generating the target image of the media file based on the original image of the media file, and effectively improve the diversity of the media file.

Acquiring an original image of the media file through the steps S202 to S208; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and adjusting the original image based on the second element information to obtain a target image of the media file. That is to say, in the embodiment, the original image of the media file based on the internet is obtained from the first element information, and then the appropriate second element information is selected based on the first element information to adjust the original image, so that the purpose of generating the target image from the original image is achieved, and the problem that the diversity of the generated image is insufficient due to the fact that the data layout information of the media file is consistent and the repetition degree of the element information is high is avoided, so that the technical problem that the image of the media file is single is solved, the technical effect of improving the diversity of the image of the media file is achieved, and the user experience is effectively improved.

The above-described method of this embodiment is further described below.

As an alternative implementation, step S204, acquiring first element information of the original image, includes: analyzing the original image to obtain a structure matrix and a visual matrix of the original image, wherein the original image comprises at least one element, the structure matrix is used for indicating the structure of the distribution of the at least one element in the original image, and the visual matrix is used for indicating the visual information presented by the at least one element in the original image.

In this embodiment, the at least one element in the original image may include, but is not limited to, a design component element such as a commodity, a document, a decoration, a logo, and the like. When obtaining the first element information of the original image is achieved, parsing the original image may be performed to parse the original image so as to parse a structure matrix and a visual matrix of the original image, where the structure matrix of the original image may be used to indicate a structure of the at least one element of the original image distributed in the original image, for example, to design a structure matrix corresponding to a position of the element, a position of the at least one element in the original image may be determined, so that a layout of the at least one element may be consistent; the visual matrix may be used to indicate visual information presented by the at least one element in the original image, and may determine a visual style, for example, the visual information may be a visual underlying characteristic of a color, a size, a texture, etc. presented by the at least one element.

As an optional implementation, parsing the original image to obtain a structure matrix and a visual matrix of the original image includes: positioning at least one element in the original image to obtain positioning information of the at least one element; and carrying out image segmentation on the original image based on the positioning information to obtain a structure matrix and a visual matrix.

In this embodiment, when the original image is analyzed, an element known in the original image may be located by using an image location technology, for example, design elements such as a commodity, a document, a decoration, and a logo in the original image are located to obtain location information, and then the original image may be segmented by using the location information of the at least one element by using an image segmentation technology to obtain the structure matrix and the visual matrix, and then, under the constraint of the structure indicated by the structure matrix and the visual angle information indicated by the visual matrix, appropriate second element information is selected, and the original image is adjusted by using the second element information, so as to obtain a target image of the media file.

As an alternative implementation manner, in step S206, acquiring the second element information corresponding to the first element information includes: determining an original background element from the first element information; and acquiring a target background element corresponding to the original background element, wherein the second element information comprises the target background element.

In this embodiment, when obtaining the second element information corresponding to the first element is implemented, an original background element may be determined from the first element information to adjust the original background element, where the original background element is used to generate a background image in the original image. Optionally, the embodiment obtains, in the target database, a target background element corresponding to the original background element, where a similarity between the target background element and the original background element is greater than a target threshold, and a target background image similar to the background image in the original image may be generated by the target background element.

As an optional implementation, obtaining the target background element corresponding to the original background element includes: acquiring at least one of the following information of the original background element: color information, depth information, and style information; recalling the targeted background element based on the at least one information.

In this embodiment, when obtaining the target background element corresponding to the original background element is implemented, color information and depth information in the original background element may be obtained first, where the color information may be a color information distribution in the original background image, and the depth information may be depth feature information in the original background image, for example, a depth feature information index, and this embodiment may recall the target background element by using the color information and the depth information together. Optionally, style information may be added to perform re-constraint, where the style information may include style features, that is, a target background element whose similarity is greater than a target threshold is recalled by using color information, depth information, and style information in the original background element together, so as to ensure accuracy of obtaining the target background element under the style constraint.

As an alternative implementation, in step S208, adjusting the original image based on the second element information to obtain the target image of the media file includes: and replacing the original background elements with the target background elements in the original image to obtain the target image.

In this embodiment, after the target background element corresponding to the original background element is obtained, the original background element may be adjusted in the original image through the target background element, and the original background element may be replaced with the target background element in the original image, so as to obtain the target image of the media file, thereby achieving the purpose of expanding the target image of the media file by using the original image of the media file as a template. Optionally, when performing the replacement processing, the embodiment may perform the replacement processing on the elements corresponding to the uniform type real object in the original image.

As an alternative implementation manner, in step S206, acquiring the second element information corresponding to the first element information includes: and acquiring the modified element information matched with the first element information, wherein the second element information comprises the modified element information.

In this embodiment, when obtaining the second element information corresponding to the first element information is implemented, obtaining decoration element information matching with the first element information may be further implemented, where the decoration element information may be used to decorate the original image based on the structure matrix of the original image to obtain the target image of the media file.

As an alternative embodiment, obtaining the decoration element information matched with the first element information includes: acquiring the position and the size of an element corresponding to the first element information in the original image; and acquiring the decoration element information matched with the position and the size.

In this embodiment, when obtaining the decoration element information matched with the first element information is implemented, the position and the size of the element corresponding to the first element information in the original image may be obtained, where the original corresponding to the first element information is at least one element in the original image. Alternatively, the embodiment may devise an aesthetic experience knowledge by sampling the location features through an algorithm of the media file hot zone bounding box to determine the location of the first element information in the original image based on the location features. The bounding box is an algorithm for solving the bounding space of the discrete point set, and can approximately replace a complex geometric object by a geometric body with a slightly larger volume and simple characteristics. Optionally, in this embodiment, the size of the element corresponding to the first element information is also determined, so that the position and size of the modified element information in the original image are inferred based on the position and size.

As an alternative embodiment, obtaining the decoration element information matched with the first element information includes: analyzing the first element information through a neural network model to obtain modified element information, wherein the neural network model is trained by using multiple groups of sample data, and each group of data in the multiple groups of sample data comprises: the label of the element information sample and the decoration element information, and the corresponding relation between the element information sample and the decoration element information.

In this embodiment, when obtaining the modifier information matching with the first element information, the first element information may be analyzed by a pre-trained neural network model, the neural Network model may be a Graph Convolutional Network model (GCN), may be referred to as a Graph Network collocation module, the training template can be analyzed according to the design template of the early designer and a plurality of groups of mass sample data, the neural network model is designed based on the training model, wherein each group of data in the multiple groups of sample data comprises a label of the element information sample and the modification element information, and a corresponding relation between the element information sample and the modification element information, the element information sample is matched with the decoration element information corresponding to the label, and the element information sample can comprise commodities, documentaries, background information, logos and the like.

As an alternative embodiment, analyzing the first element information through the neural network model to obtain modified element information, including: analyzing at least one of the following information in the first element information through a neural network model to obtain modification element information: article information, case information, identification information, original background elements.

In this embodiment, the first element information includes article information, copy information, identification information, and original background element, where the article information is used to indicate an article to be sold, the copy information may be introduction information of the article to be sold, the identification information may be a logo of the article to be sold, and the original background element may be a background of the article to be sold, and the above information is analyzed through a pre-trained neural network model to obtain decoration element information matching therewith, and the information may include information such as a position of a decoration element to be added to an original image based on a structural matrix of the original image, and a size of the decoration element.

As an optional implementation, the decoration element information includes: and the decoration elements are used for decorating the target area of the original image, and/or the main body decoration elements are used for decorating the target object in the original image.

In this embodiment, the decoration element information may be a decoration element, which may be a decoration element for decorating a target area of the original image, for example, a decoration element with a certain shape and a certain color, which may include a copy background, and the decoration element may be added to the target area of the original image on the basis of a structural matrix of the original image to obtain a target image of the media file, where the target area may be any area that may be decorated in the original image.

Optionally, the modification element of this embodiment may also be a body modification element, and the body modification element may be configured to modify a target object in the original image, where the target object may be an object in the original image that needs to be highlighted, for example, an article that needs to be sold, a model that displays the article that needs to be sold, or the like, and is not limited herein.

Alternatively, the body modifying element of this embodiment may be displayed below the dotted modifying element, that is, the dotted modifying element may cover a portion of the body modifying element.

As an alternative implementation, in step S208, adjusting the original image based on the second element information to obtain the target image of the media file includes: and adding the decoration element information to the original image to obtain a target image.

In this embodiment, when the original image is adjusted based on the second element information, the interspersing decoration element and/or the main body decoration element may be added on the basis of the structural matrix of the original image, and may be placed in a suitable position area in the original image, and may be fine-tuned again, for example, user interaction adjustment may be implemented through a document model, so as to obtain a target image of the media file, and further, the purpose of expanding the target image of the media file by using the original image of the media file as a template is achieved.

The present invention also provides another image processing method from the user interaction perspective.

Fig. 3 is a flowchart of another image processing method according to an embodiment of the present invention. As shown in fig. 3, the method may include the steps of:

step S302, displaying the original image of the media file on the operation interface.

In the technical solution provided in step S302 of the present invention, a media file is input to the terminal, and an original image of the media file is further displayed on the operation interface, wherein the media file is a media file for which a new image needs to be expanded, and may be a banner advertisement. Optionally, the media file may be a media file in an e-market scene, for example, a shopping advertisement in the e-market scene.

The original image of this embodiment is the most initial image of the media file, and may be an original banner image, including the original image content of the media file.

Step S304, reading first element information of the original image, and acquiring second element information corresponding to the first element information.

In the technical solution provided in step S304 of the present invention, after an original image of a media file is displayed on an operation interface, first element information of the original image is read, and second element information corresponding to the first element information is obtained, where a similarity between the second element information and the first element information is greater than a target threshold.

In this embodiment, the first element information read from the original image is information of the design constituent elements of the original image, and may include information of positions, sizes, types, and the like of the design constituent elements, and the first element information may include element information that needs to be replaced, and may also include element information that needs to be decorated.

In this embodiment, the second element information and the first element information have a corresponding relationship, the second element information may be obtained in the target database based on the first element information, the similarity between the second element information and the first element information is greater than a target threshold, and the similarity between the feature of the second element information and the feature of the first element information may be greater than the target threshold, where the feature may be any one or more of a color feature, a depth information feature, a style feature, and the like, so that the second element information is element information suitable for replacing or modifying the first element information, so that the similarity between the target image of the media file obtained by adjusting the original image based on the second element information and the original image is greater.

Step S306, displaying a target image of the media file obtained by adjusting the original image based on the second element information in the browsing area of the operation interface.

In the technical solution provided in step S306 of the present invention, after the second element information corresponding to the first element information is acquired, the target image of the media file obtained by adjusting the original image based on the second element information is displayed in the browsing area of the operation interface. The purpose of adjusting the original image by interacting with the user can be achieved on the operation interface.

According to the embodiment, on the basis of the original image, the target image of the media file is expanded through the second element information and then displayed in the browsing area of the operation interface, the visual similarity between the target image and the original image is smaller than a certain threshold value, namely, the target image is at least one image which is visually similar to the original image and can be called as a result image for adjusting the original image, so that the purpose of generating the target image of the media file based on the original image of the media file is achieved, and the diversity of the media file is effectively improved.

As an alternative implementation, step S304, reading the first element information of the original image, includes: reading a structure matrix and a visual matrix of an original image, wherein the original image comprises at least one element, the structure matrix is used for indicating the structure of the at least one element distributed in the original image, and the visual matrix is used for indicating the visual information presented by the at least one element in the original image.

In this embodiment, the at least one element read from the original image may include, but is not limited to, a design component element such as a commodity, a pattern, a decoration, a logo, and the like. The embodiment may perform parsing on the original image to parse a structure matrix and a visual matrix of the original image, and read the structure matrix and the visual matrix, where the structure matrix may be used to indicate a structure of the at least one element of the original image distributed in the original image, and the visual matrix may be used to indicate visual information presented by the at least one element in the original image.

As an alternative implementation manner, in step S304, acquiring the second element information corresponding to the first element information includes: determining an original background element from the first element information; and acquiring a target background element corresponding to the original background element, wherein the second element information comprises the target background element.

In this embodiment, the original background element is determined from the first element information to adjust the original background element. Optionally, the embodiment obtains, in the target database, a target background element corresponding to the original background element, where a similarity between the target background element and the original background element is greater than a target threshold, and a target background image similar to the background image in the original image may be generated by the target background element.

As an optional implementation, the method further comprises: and replacing the original background elements with the target background elements in the original image to obtain the target image.

In this embodiment, after the target background element corresponding to the original background element is obtained, the original background element may be replaced with the target background element in the original image, so as to obtain a target image of the media file, and then the target image is displayed in the browsing area of the display interface, thereby achieving the purpose of expanding the target image of the media file by using the original image of the media file as a template.

As an alternative implementation manner, in step S304, acquiring the second element information corresponding to the first element information includes: and acquiring the modified element information matched with the first element information, wherein the second element information comprises the modified element information.

In this embodiment, modifier information matching the first element information is obtained, and the modifier information may be used to modify the original image based on the structural matrix of the original image to obtain the target image of the media file.

As an optional implementation, the method further comprises: and adding the decoration element information to the original image to obtain a target image.

In the embodiment, the embellishment decoration elements and/or the main body decoration elements in the decoration element information are added on the basis of the structural matrix of the original image, so that the target image of the media file is obtained, and the purpose of expanding the target image of the media file by taking the original image of the media file as a template is further achieved.

The embodiment of the invention also provides another image processing method applied to the E-commerce scene.

Fig. 4 is a flowchart of another image processing method according to an embodiment of the present invention. As shown in fig. 4, the method may include the steps of:

step S402, acquiring an original image of the media file, wherein the original image comprises an item to be sold.

In the technical solution provided by step S402 of the present invention, the media file may be a media file in an e-market scene, for example, a shopping advertisement in the e-market scene, and may be a banner. The original image comprises an article (commodity) to be sold in the e-market scene, and the media file is a media file for which a new image needs to be developed.

The image of the media file of this embodiment may be an e-commerce advertisement image, and an original image of the media file is obtained, where the original image is the most initial image of the media file and may be used as a template image of an image to be developed, where the developed image still includes an article to be sold.

In step S404, first element information of the original image is acquired.

In the technical solution provided by step S404 of the present invention, after the original image of the media file is acquired, the first element information of the original image is acquired.

In this embodiment, the first element information is information of a design constituent element of the original image, and may include information of a position, a size, a type, and the like of the design constituent element. Optionally, the first element information of this embodiment includes element information that needs to be replaced, and may also include element information that needs to be decorated.

Step S406, second element information corresponding to the first element information is obtained, wherein the similarity between the second element information and the first element information is greater than a target threshold value and is used for modifying the article.

In the technical solution provided in step S406 of the present invention, a corresponding relationship exists between the second element information and the first element information, the second element information may be obtained in the target database based on the first element information, a similarity between the second element information and the first element information is greater than a target threshold, and a similarity between a feature of the second element information and a feature of the first element information may be greater than a target threshold, where the second element information is element information suitable for replacing or modifying the first element information, so that a similarity between a target image and an original image of a media file obtained by adjusting the original image based on the second element information is greater, and a purpose of modifying an article to be sold is achieved.

Step S408, adjusting the original image based on the second element information to obtain a target image of the media file.

In the technical solution provided in step S408 of the present invention, after the second element information corresponding to the first element information is obtained, the original image is adjusted based on the second element information, that is, on the basis of the original image, the target image of the media file is expanded through the second element information, the target image still includes the article to be sold, and the degree of visual similarity to the original image is smaller than a certain threshold. Optionally, the target image is at least one image visually similar to the original image, which may be referred to as a result image of adjusting the original image, so as to achieve the purpose of generating the target image of the media file based on the original image of the media file, and effectively improve the diversity of the media file.

As an alternative implementation, step S404, acquiring first element information of the original image, including: analyzing the original image to obtain a structure matrix and a visual matrix of the original image, wherein the original image comprises at least one element, the structure matrix is used for indicating the structure of the distribution of the at least one element in the original image, and the visual matrix is used for indicating the visual information presented by the at least one element in the original image.

In this embodiment, the at least one element in the original image may include, but is not limited to, a design component element such as a commodity, a file, a decoration, a logo, etc. in the e-market scene. The embodiment may be that an original image of a media file in an e-market scene is parsed to obtain a structure matrix and a visual matrix of the original image, where the structure matrix may be used to indicate a structure of the at least one element of the original image distributed in the original image, and the visual matrix may be used to indicate visual information presented by the at least one element in the original image.

As an alternative implementation manner, in step S406, acquiring second element information corresponding to the first element information includes: determining an original background element from the first element information; and acquiring a target background element corresponding to the original background element, wherein the second element information comprises the target background element.

In this embodiment, the original background element is determined from the first element information to adjust the original background element to highlight the item to be vended. Optionally, the embodiment obtains, in the target database, a target background element corresponding to the original background element, where a similarity between the target background element and the original background element is greater than a target threshold, and may generate, by the target background element, a target background image similar to the background image in the original image, so as to highlight the article to be sold in the target image.

As an optional implementation manner, in step S408, adjusting the original image based on the second element information to obtain a target image of the media file includes: and replacing the original background elements with the target background elements in the original image to obtain the target image.

In this embodiment, after the target background element corresponding to the original background element is acquired, the original background element may be directly replaced by the target background element in the original image, so as to obtain the target image of the media file.

As an alternative implementation manner, in step S406, acquiring second element information corresponding to the first element information includes: and acquiring the modified element information matched with the first element information, wherein the second element information comprises the modified element information.

As an optional implementation manner, in step S408, adjusting the original image based on the second element information to obtain a target image of the media file includes: and adding the decoration element information to the original image to obtain a target image.

In the embodiment, the interspersing decoration elements and/or the main body decoration elements are/is added on the basis of the structural matrix of the original image, so that the target image of the media file is obtained, and the purpose of expanding the target image of the media file by taking the original image of the media file as a template is further realized.

The embodiment of the invention also provides another image processing method from the user interaction angle.

The image processing method of this embodiment includes: displaying an original image of the media file on an operation interface; responding to an input operation instruction acting on the operation interface, and acquiring second element information; reading first element information of an original image corresponding to second element information, wherein the similarity between the second element information and the first element information is larger than a target threshold value; and displaying a target image of the media file obtained by adjusting the original image based on the second element information in the browsing area of the operation interface.

In this embodiment, the operation interface may initially display an original image of a media file, where the media file is a media file that needs to be expanded to a new image, and may be a banner advertisement. Optionally, the media file may be a media file in an e-market scene, for example, a shopping advertisement in the e-market scene. The original image of this embodiment is the most initial image of the media file, and may be an original banner image, including the original image content of the media file.

In this embodiment, the second element information is obtained in response to an input operation instruction acted on the operation interface by a user, where the input operation instruction may include information input by the user for modifying the original image, and may be information input by the user for a feature of modifying the original image. After the second element information is acquired, first element information corresponding to the second element information may be read from the original image, where the first element information is information of design constituent elements of the original image, and may include information of positions, sizes, types, and the like of the design constituent elements, and the first element information may include element information that needs to be replaced by the second element information, and may also include element information that needs to be decorated by the second element information.

In this embodiment, the second element information and the first element information have a corresponding relationship, and the similarity between the second element information and the first element information is greater than a target threshold, that is, the first element information whose similarity between the features and the first element information is greater than the target threshold may be read from the original image based on the second element information, where the features may be any one or more of color features, depth information features, style features, and the like, so that the second element information is element information suitable for replacing or modifying the first element information, and the similarity between the target image of the media file obtained by adjusting the original image based on the second element information and the original image is greater.

According to the embodiment, on the basis of the original image, the target image of the media file is expanded through the second element information obtained based on the input operation instruction of the user, and then the target image is displayed in the browsing area of the operation interface, the visual similarity between the target image and the original image is smaller than a certain threshold value, namely, the target image is at least one image which is visually similar to the original image and can be called as a result image for adjusting the original image, so that the purpose of generating the target image of the media file based on the original image of the media file is achieved, and the diversity of the media file is effectively improved.

The above-described method of this embodiment is further described below.

As an optional implementation, the obtaining the second element information includes: and acquiring target characteristics carried by the input operation instruction, and reading second element information corresponding to the target characteristics.

In this embodiment, the input operation instruction triggered by the user may carry a target feature matching with a user requirement, where the target feature may be used to indicate a characteristic of a portion that needs to be adjusted, for example, if the user is unsatisfied with at least one of target features such as some colors, some styles, and current depth information in the original image, the input operation instruction carrying at least one of the target features such as some colors, some styles, and current depth information may be triggered, and then second element information that does not include the target feature may be read, where the second element information may be used to adjust first element information corresponding to some colors and some styles in the original image, so as to obtain a target image of the media file, where the target image does not include some colors, some styles, and some styles that are not liked by the user, At least one of the current depth information.

Optionally, the target feature of this embodiment may also be a feature allowed to be possessed by the original image after adjustment, and meet a user requirement, for example, the target feature is at least one of a color, a style, depth information, and the like allowed to be possessed by the original image after adjustment, and if the user desires that the original image after adjustment may include at least one of a favorite color, a favorite style, and required depth information, an input operation instruction carrying at least one of the target features of the color, the style, the depth information, and the like allowed to be possessed by the original image after adjustment may be triggered, and then second element information including the target feature may be read, and the second element information may be used to adjust at least one of a corresponding original color, an original style, and original depth information in the original image, so as to obtain a target image of the media file, the target image includes at least one of a favorite color, a favorite style, and required depth information of the user.

As an optional implementation, reading the second element information corresponding to the target feature includes at least one of: selecting a target background element from at least one element information corresponding to the target feature, wherein the target background element corresponds to an original background element in the first element information, and the second element information comprises the target background element; and selecting modification element information from at least one element information corresponding to the target feature, wherein the modification element information is matched with the first element information, and the second element information comprises the modification element information.

In this embodiment, the similarity between the feature of each of the at least one type of elemental information corresponding to the target feature and the feature of the first elemental information is greater than the target threshold. When the second element information corresponding to the target feature is read, the target background element may be selected from the at least one element information. The embodiment can select a target background element from at least one element information in the target database, wherein the similarity between the characteristics of the target background element and the characteristics of the original background element in the original image is greater than a target threshold, and a target background image similar to the background image in the original image can be generated through the target background element, so that an article to be sold is highlighted in the target image. Optionally, in the original image, the original background element is replaced with the target background element, so as to obtain the target image.

In this embodiment, when reading the second element information corresponding to the target feature, modification element information may be selected from at least one element information, and the modification element information may be used to modify the original image based on the structural matrix of the original image to obtain the target image of the media file. Optionally, in this embodiment, a dotted decoration element and/or a main body decoration element is added on the basis of the structural matrix of the original image, so as to obtain the target image of the media file, and further, the purpose of expanding the target image of the media file by using the original image of the media file as a template is achieved.

As an alternative embodiment, the at least one type of element information is element information of at least one style of image.

In the embodiment, the user can adjust the original image according to the required style, and the at least one piece of element information can be the element information of the image with the at least one style corresponding to the target feature, so that the user can input the original image in a targeted manner according to the characteristic of modifying the original image, the required style of the image is flexibly selected, and the diversity and the user experience of the media file are effectively improved.

The image processing method of the embodiment can be a copy generation scheme in a shopping mall scene, and can obtain a structure matrix and a visual matrix of a media file by means of analyzing and processing an original image of a massive media file of the internet, and then select appropriate element information based on the structure matrix and the visual matrix to replace or modify elements in the original image to obtain a target image, so that the purpose of generating the target image from the original image is achieved, and the diversity and the user experience of the media file are effectively improved.

Example 2

The following is an illustration of a preferred implementation of this embodiment of the invention. Specifically, a method for generating a banner image in an e-commerce scene is described as an example.

In the related art, under the electronic market scene, the data layout information is consistent, and the element information repetition degree is high, so that the generated images are not diverse enough, and the user experience is poor.

In the embodiment, massive Banner images of the Internet can be used for analyzing, a Banner structure matrix and a visual matrix are obtained, then under the structural and visual constraints, appropriate element information can be selected to replace the original Banner content, or modification elements are added on the basis of the structure matrix, and the generation of a graph from one graph to another is carried out, so that the diversity and the user experience of the Banner are effectively improved.

The method of this embodiment is further described below.

Fig. 5 is a flowchart of another method of generating an image of a media file according to an embodiment of the present invention. As shown in fig. 5, the method may include the steps of:

step S501, inputting an original image of the banner, and analyzing the original image.

In this embodiment, an original image of a banner is input, elements in the original image are located, for example, design constituent elements such as a commodity, document information, a decoration element, and a logo are located by using an image location technology, and then a structure matrix and a visual matrix of the banner are determined by using an image segmentation technology, which may include visual information of each element.

Fig. 6A is a schematic diagram of an original image of a banner according to an embodiment of the present invention. As shown in fig. 6A, the original image includes a model showing a T-shirt of clothes to be sold, and the pattern information "a-type female T-shirt", "instant participation", and the like, which are not limited in detail herein.

Fig. 6B is a schematic diagram of a structural matrix according to an embodiment of the invention. As shown in fig. 6B, by analyzing the original image shown in fig. 6A, the structural matrix can be obtained, and the positions of the elements in fig. 6A in the original image can be indicated.

Step S502, the original background element in the original image is replaced by the target background element.

This embodiment may replace the original background elements of the original image of the banner with target background elements. Optionally, the embodiment recalls the target background element by means of the image color information distribution and depth feature information index of the background image of the original image of the banner.

Optionally, the embodiment may adopt that the image color information distribution and the depth feature information are recalled together, and may also add style feature re-constraint, thereby ensuring the accuracy of recalling the target background element under the constraint of the style feature.

In step S503, after replacing the original background element in the original image with the target background element, adding a embellishing element by the embellishing module to obtain a first result image.

In this embodiment, the modification module, i.e. the modification element module, may add a dotted modification element in the original image, which may include a pattern shading. Optionally, the embodiment determines the position of the interspersed modification element, which may be obtained using a GCN collocation network. Optionally, in this embodiment, the GCN collocation network is designed based on a training template that is designed by a previous designer and analyzed from mass data. The embodiment can deduce the modification elements matched with the original image through the commodity, the cultural relic and the background information in the Banner original image.

Fig. 7A is a schematic diagram of adding a dotted decoration element on the basis of a structural matrix according to an embodiment of the present invention. As shown in fig. 7A, the first result image shown in fig. 7B is generated by adding the decoration elements a1, a2, a3 and a4 to fig. 6B, wherein fig. 7B is a schematic diagram of the first result image obtained by adding the decoration elements to a structural matrix according to an embodiment of the present invention. As shown in fig. 7B, the display information a1 in the first result image corresponds to the dotted-decoration element a1, the display information a2 in the first result image corresponds to the dotted-decoration element a2, the display information A3 in the first result image corresponds to the dotted-decoration element A3, and the display information a4 in the first result image corresponds to the dotted-decoration element a 4.

Step S504, the interactive adjustment with the user is realized through the file module.

The case model of the embodiment can be used for realizing interactive adjustment with a user so as to realize similar result images developed by the original images based on the banner according to the requirements of the user.

Step S505, adding a main body decoration element to the first result image through the decoration module, to obtain a second result image.

The embodiment can further add the subject decoration element in the two result images, and can determine the position characteristic through a Banner hot zone bounding box algorithm by using aesthetic experience knowledge, thereby determining the position of adding the subject decoration element in the original image. Optionally, this embodiment may use other design element size collocation in the original image of the banner to infer the size of the subject modifier element.

Fig. 8A is a schematic diagram of adding a body modifying element on the basis of a structural matrix according to an embodiment of the present invention. As shown in fig. 8A, a subject modifying element B1 may be added on the structural matrix based on fig. 7A, so as to obtain a second resulting image shown in fig. 8B. Fig. 8B is a schematic diagram of a second result image obtained by adding a body modifying element to a structural matrix according to an embodiment of the present invention. As shown in fig. 8B, the display information B1 in the second result image, which is the result image similar to the original image based on the banner finally developed, corresponds to the subject modification element B1. Alternatively, the body modifying element of this embodiment may be displayed below the dotted modifying element, that is, the dotted modifying element may cover a portion of the body modifying element.

It should be noted that, in this embodiment, the entire banner copy generation mode can be realized by the above method, and N visually similar target images of the banner are developed by using one original image of the banner as a template.

Fig. 9 is a schematic diagram of an interactive interface of an image processing method according to an embodiment of the present invention. As shown in fig. 9, a user may drag an original image of the banner into an "add" text box, read first element information of the original image by clicking an "image generation" key, and obtain second element information corresponding to the first element information, where a similarity between the second element information and the first element information is greater than a target threshold, and finally, in a browsing area of an operation interface, display at least one target image of the banner obtained by adjusting the original image based on the second element information, thereby avoiding that the generated images are not sufficient in diversity due to relatively consistent data layout information of the media files and relatively high repetition of the element information, thereby solving a technical problem that the images of the media files are relatively single, and achieving a technical effect of improving the diversity of the images of the media files.

Fig. 10 is a schematic diagram of advertisement image generation based on image parsing in an e-market scene according to an embodiment of the present invention. As shown in fig. 10, an original image of a banner is input to a computer device, the computer device reads first element information of the original image, and acquires second element information corresponding to the first element information, optionally acquires a target background element for replacing the original background element of the original image, and deduces, through a GCN collocation network, modifier element information for modifying the original image. The similarity between the second element information and the first element information is larger than the target threshold, the target image of the banner obtained by adjusting the original image through the second element information is output, and finally the target image of the banner obtained by adjusting the original image based on the second element information is displayed in the browsing area of the operation interface, so that the problem that the diversity of the generated images is insufficient due to the fact that the data layout information of the media files is consistent and the repetition degree of the element information is high is solved, the technical problem that the images of the media files are single is solved, and the technical effect of improving the diversity of the images of the media files is achieved.

The image processing method of the embodiment can be a copying generation scheme in a shopping mall scene, can obtain a structural matrix and a visual matrix of the banner by means of analyzing massive internet banner original images, and then selects appropriate element information based on the structural matrix and the visual matrix to replace or modify elements in the banner original images to obtain target images of the banner, achieves the purpose of generating the target images of the banner from the original images, and effectively improves diversity and user experience of media files.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.

Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.

Example 3

According to an embodiment of the present invention, there is also provided an image processing apparatus for implementing the image processing method shown in fig. 2 described above.

Fig. 11 is a schematic diagram of an image processing apparatus according to an embodiment of the present invention. As shown in fig. 11, the image processing apparatus 110 may include: a first acquisition unit 111, a second acquisition unit 112, a third acquisition unit 113, and an adjustment unit 114.

A first obtaining unit 111 for obtaining an original image of the media file.

A second obtaining unit 112, configured to obtain first element information of the original image.

A third obtaining unit 113, configured to obtain second element information corresponding to the first element information, where a similarity between the second element information and the first element information is greater than a target threshold.

And an adjusting unit 114, configured to adjust the original image based on the second element information, so as to obtain a target image of the media file.

It should be noted here that the first acquiring unit 111, the second acquiring unit 112, the third acquiring unit 113, and the adjusting unit 114 correspond to steps S202 to S208 in embodiment 1, and the four units are the same as the corresponding steps in the implementation example and application scenario, but are not limited to the disclosure in the first embodiment. It should be noted that the above units as a part of the apparatus may operate in the computer terminal 10 provided in the first embodiment.

According to an embodiment of the present invention, there is also provided an image processing apparatus for implementing the image processing method shown in fig. 3 described above.

Fig. 12 is a schematic diagram of another image processing apparatus according to an embodiment of the present invention. As shown in fig. 12, the image processing apparatus 120 may include: a first display unit 121, a reading unit 122, and a second display unit 123.

A first display unit 121 for displaying an original image of the media file on the operation interface.

The reading unit 122 is configured to read first element information of an original image, and acquire second element information corresponding to the first element information, where a similarity between the second element information and the first element information is greater than a target threshold.

And a second display unit 123, configured to display, in the browsing area of the operation interface, a target image of the media file obtained by adjusting the original image based on the second element information.

It should be noted here that the first display unit 121, the reading unit 122, and the second display unit 123 correspond to steps S302 to S306 in embodiment 1, and the three units are the same as the examples and application scenarios realized by the corresponding steps, but are not limited to the disclosure of the first embodiment. It should be noted that the above units as a part of the apparatus may operate in the computer terminal 10 provided in the first embodiment.

Fig. 13 is a schematic diagram of another image processing apparatus according to an embodiment of the present invention. As shown in fig. 13, the image processing apparatus 130 may include: a fourth acquiring unit 131, a fifth acquiring unit 132, a sixth acquiring unit 133, and a second adjusting unit 134.

A fourth obtaining unit 131, configured to obtain an original image of the media file, where the original image includes an item to be sold;

a fifth acquiring unit 132 configured to acquire first element information of the original image;

a sixth obtaining unit 133, configured to obtain second element information corresponding to the first element information, where a similarity between the second element information and the first element information is greater than a target threshold, and the second obtaining unit is configured to modify the article;

a second adjusting unit 134, configured to adjust the original image based on the second element information, so as to obtain a target image of the media file.

It should be noted here that the fourth acquiring unit 131, the fifth acquiring unit 132, the sixth acquiring unit 133 and the second adjusting unit 134 correspond to steps S402 to S408 in embodiment 1, and the four units are the same as the corresponding steps in the implementation example and the application scenario, but are not limited to the disclosure in the first embodiment. It should be noted that the above units as a part of the apparatus may operate in the computer terminal 10 provided in the first embodiment.

The image processing device of the embodiment obtains the first element information from the original image of the media file based on the internet, and then selects the appropriate second element information based on the first element information to adjust the original image, so that the purpose of generating the target image from the original image is achieved, the problem that the generated image is not diverse enough due to the fact that the data layout information of the media file is consistent and the repetition degree of the element information is high is avoided, the technical problem that the image of the media file is single is solved, and the technical effect of improving the diversity of the image of the media file is achieved.

Example 3

The embodiment of the invention can provide a computer terminal which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced with a terminal device such as a mobile terminal.

Optionally, in this embodiment, the computer terminal may be located in at least one network device of a plurality of network devices of a computer network.

In this embodiment, the computer terminal may execute program codes of the following steps in the image processing method of the application program: acquiring an original image of a media file; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and adjusting the original image based on the second element information to obtain a target image of the media file.

Alternatively, fig. 14 is a block diagram of a mobile terminal according to an embodiment of the present invention. As shown in fig. 14, the mobile terminal a may include: one or more processors 142 (only one shown), a memory 144, and a transmission device 146.

The transmission device is used for acquiring the original image of the media file acquired by the image acquisition device.

The memory may be configured to store software programs and modules, such as program instructions/modules corresponding to the image processing method and apparatus in the embodiments of the present invention, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, so as to implement the image processing method. The memory may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory located remotely from the processor, which may be connected to the mobile terminal a via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: acquiring an original image of a media file; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and adjusting the original image based on the second element information to obtain a target image of the media file.

Optionally, the processor may further execute the program code of the following steps: acquiring first element information of an original image, comprising: analyzing the original image to obtain a structure matrix and a visual matrix of the original image, wherein the original image comprises at least one element, the structure matrix is used for indicating the structure of the distribution of the at least one element in the original image, and the visual matrix is used for indicating the visual information presented by the at least one element in the original image.

Optionally, the processor may further execute the program code of the following steps: positioning at least one element in the original image to obtain positioning information of the at least one element; and carrying out image segmentation on the original image based on the positioning information to obtain a structure matrix and a visual matrix.

Optionally, the processor may further execute the program code of the following steps: determining an original background element from the first element information; and acquiring a target background element corresponding to the original background element, wherein the second element information comprises the target background element.

Optionally, the processor may further execute the program code of the following steps: acquiring at least one of the following information of the original background element: color information, depth information, and style information; recalling the targeted background element based on the at least one information.

Optionally, the processor may further execute the program code of the following steps: and replacing the original background elements with the target background elements in the original image to obtain the target image.

Optionally, the processor may further execute the program code of the following steps: and acquiring the modified element information matched with the first element information, wherein the second element information comprises the modified element information.

Optionally, the processor may further execute the program code of the following steps: acquiring the position and the size of an element corresponding to the first element information in the original image; and acquiring the decoration element information matched with the position and the size.

Optionally, the processor may further execute the program code of the following steps: analyzing the first element information through a neural network model to obtain modified element information, wherein the neural network model is trained by using multiple groups of sample data, and each group of data in the multiple groups of sample data comprises: the label of the element information sample and the decoration element information, and the corresponding relation between the element information sample and the decoration element information.

Optionally, the processor may further execute the program code of the following steps: analyzing at least one of the following information in the first element information through a neural network model to obtain modification element information: article information, case information, identification information, original background elements.

Optionally, the processor may further execute the program code of the following steps: and adding the decoration element information to the original image to obtain a target image.

As an alternative example, the processor may invoke the information stored in the memory and the application program via the transmission means to perform the following steps: displaying an original image of the media file on an operation interface; reading first element information of an original image, and acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and displaying a target image of the media file obtained by adjusting the original image based on the second element information in the browsing area of the operation interface.

Optionally, the processor may further execute the program code of the following steps: reading first element information of an original image, including: reading a structure matrix and a visual matrix of an original image, wherein the original image comprises at least one element, the structure matrix is used for indicating the structure of the at least one element distributed in the original image, and the visual matrix is used for indicating the visual information presented by the at least one element in the original image.

As an alternative example, the processor may invoke the information stored in the memory and the application program via the transmission means to perform the following steps: acquiring an original image of a media file, wherein the original image comprises an item to be sold; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value and is used for modifying the article; and adjusting the original image based on the second element information to obtain a target image of the media file.

Optionally, the processor may further execute the program code of the following steps: analyzing the original image to obtain a structure matrix and a visual matrix of the original image, wherein the original image comprises at least one element, the structure matrix is used for indicating the structure of the distribution of the at least one element in the original image, and the visual matrix is used for indicating the visual information presented by the at least one element in the original image.

As an alternative example, the processor may invoke the information stored in the memory and the application program via the transmission means to perform the following steps: displaying an original image of the media file on an operation interface; responding to an input operation instruction acting on the operation interface, and acquiring second element information; reading first element information of an original image corresponding to second element information, wherein the similarity between the second element information and the first element information is larger than a target threshold value; and displaying a target image of the media file obtained by adjusting the original image based on the second element information in the browsing area of the operation interface.

Optionally, the processor may further execute the program code of the following steps: and acquiring target characteristics carried by the input operation instruction, and reading second element information corresponding to the target characteristics.

Optionally, the processor may further execute program code for at least one of: selecting a target background element from at least one element information corresponding to the target feature, wherein the target background element corresponds to an original background element in the first element information, and the second element information comprises the target background element; and selecting modification element information from at least one element information corresponding to the target feature, wherein the modification element information is matched with the first element information, and the second element information comprises the modification element information.

The embodiment of the invention provides an image processing method. Acquiring an original image of a media file; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and adjusting the original image based on the second element information to obtain a target image of the media file. That is to say, in the embodiment, the first element information is obtained from the original image of the media file based on the internet, and then the appropriate second element information is selected based on the first element information to adjust the original image, so that the purpose of generating the target image from the original image is achieved, and the problem that the diversity of the generated image is insufficient due to the fact that the data layout information of the media file is consistent and the repetition degree of the element information is high is avoided, so that the technical problem that the image of the media file is single is solved, and the technical effect of improving the diversity of the image of the media file is achieved.

It can be understood by those skilled in the art that the structure shown in fig. 14 is only an illustration, and the Mobile terminal a may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 14 is not intended to limit the structure of the mobile terminal a. For example, mobile terminal a may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in fig. 14, or have a different configuration than shown in fig. 14.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

Example 4

The embodiment of the invention also provides a storage medium. Optionally, in this embodiment, the storage medium may be configured to store a program code executed by the image processing method provided in the first embodiment.

Optionally, in this embodiment, the storage medium may be located in any one of computer terminals in a computer terminal group in a computer network, or in any one of mobile terminals in a mobile terminal group.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring an original image of a media file; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and adjusting the original image based on the second element information to obtain a target image of the media file.

Optionally, the storage medium is further arranged to store program code for performing the steps of: analyzing the original image to obtain a structure matrix and a visual matrix of the original image, wherein the original image comprises at least one element, the structure matrix is used for indicating the structure of the distribution of the at least one element in the original image, and the visual matrix is used for indicating the visual information presented by the at least one element in the original image.

Optionally, the storage medium is further arranged to store program code for performing the steps of: positioning at least one element in the original image to obtain positioning information of the at least one element; and carrying out image segmentation on the original image based on the positioning information to obtain a structure matrix and a visual matrix.

Optionally, the storage medium is further arranged to store program code for performing the steps of: determining an original background element from the first element information; and acquiring a target background element corresponding to the original background element, wherein the second element information comprises the target background element.

Optionally, the storage medium is further arranged to store program code for performing the steps of: acquiring at least one of the following information of the original background element: color information, depth information, and style information; recalling the targeted background element based on the at least one information.

Optionally, the storage medium is further arranged to store program code for performing the steps of: and replacing the original background elements with the target background elements in the original image to obtain the target image.

Optionally, the storage medium is further arranged to store program code for performing the steps of: and acquiring the modified element information matched with the first element information, wherein the second element information comprises the modified element information.

Optionally, the storage medium is further arranged to store program code for performing the steps of: acquiring the position and the size of an element corresponding to the first element information in the original image; and acquiring the decoration element information matched with the position and the size.

Optionally, the storage medium is further arranged to store program code for performing the steps of: analyzing the first element information through a neural network model to obtain modified element information, wherein the neural network model is trained by using multiple groups of sample data, and each group of data in the multiple groups of sample data comprises: the label of the element information sample and the decoration element information, and the corresponding relation between the element information sample and the decoration element information.

Optionally, the storage medium is further arranged to store program code for performing the steps of: analyzing at least one of the following information in the first element information through a neural network model to obtain modification element information: article information, case information, identification information, original background elements.

Optionally, the storage medium is further arranged to store program code for performing the steps of: and adding the decoration element information to the original image to obtain a target image.

As another alternative example, the storage medium is arranged to store program code for performing the steps of: displaying an original image of the media file on an operation interface; reading first element information of an original image, and acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and displaying a target image of the media file obtained by adjusting the original image based on the second element information in the browsing area of the operation interface.

Optionally, the storage medium is further arranged to store program code for performing the steps of: reading a structure matrix and a visual matrix of an original image, wherein the original image comprises at least one element, the structure matrix is used for indicating the structure of the at least one element distributed in the original image, and the visual matrix is used for indicating the visual information presented by the at least one element in the original image.

As another alternative example, the storage medium is arranged to store program code for performing the steps of: acquiring an original image of a media file, wherein the original image comprises an item to be sold; acquiring first element information of an original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value and is used for modifying the article; and adjusting the original image based on the second element information to obtain a target image of the media file.

As another alternative example, the storage medium is arranged to store program code for performing the steps of: displaying an original image of the media file on an operation interface; responding to an input operation instruction acting on the operation interface, and acquiring second element information; reading first element information of an original image corresponding to second element information, wherein the similarity between the second element information and the first element information is larger than a target threshold value; and displaying a target image of the media file obtained by adjusting the original image based on the second element information in the browsing area of the operation interface.

Optionally, the storage medium is further arranged to store program code for performing the steps of: and acquiring target characteristics carried by the input operation instruction, and reading second element information corresponding to the target characteristics.

Optionally, the storage medium is further arranged to store program code for performing at least one of the following: selecting a target background element from at least one element information corresponding to the target feature, wherein the target background element corresponds to an original background element in the first element information, and the second element information comprises the target background element; and selecting modification element information from at least one element information corresponding to the target feature, wherein the modification element information is matched with the first element information, and the second element information comprises the modification element information.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. An image processing method, comprising:

acquiring an original image of a media file;

acquiring first element information of the original image;

acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value;

and adjusting the original image based on the second element information to obtain a target image of the media file.

2. The method of claim 1, wherein obtaining first element information of the original image comprises:

analyzing the original image to obtain a structure matrix and a visual matrix of the original image, wherein the original image comprises at least one element, the structure matrix is used for indicating the structure of the at least one element distributed in the original image, and the visual matrix is used for indicating the visual information presented by the at least one element in the original image.

3. The method of claim 2, wherein parsing the original image to obtain a structural matrix and a visual matrix of the original image comprises:

positioning the at least one element in the original image to obtain positioning information of the at least one element;

and carrying out image segmentation on the original image based on the positioning information to obtain the structural matrix and the visual matrix.

4. The method of claim 1, wherein obtaining second element information corresponding to the first element information comprises:

determining an original background element from the first element information;

and acquiring a target background element corresponding to the original background element, wherein the second element information comprises the target background element.

5. The method of claim 4, wherein obtaining the target background element corresponding to the original background element comprises:

acquiring at least one of the following information of the original background element: color information, depth information, and style information;

recalling the target context element based on the at least one information.

6. The method of claim 4, wherein adjusting the original image based on the second element information to obtain a target image of the media file comprises:

and replacing the original background elements with the target background elements in the original image to obtain the target image.

7. The method of claim 1, wherein obtaining second element information corresponding to the first element information comprises:

and acquiring the modified element information matched with the first element information, wherein the second element information comprises the modified element information.

8. The method of claim 7, wherein obtaining modifier information that matches the first element information comprises:

acquiring the position and the size of an element corresponding to the first element information in the original image;

acquiring the decoration element information matched with the position and the size.

9. The method of claim 7, wherein obtaining modifier information that matches the first element information comprises:

analyzing the first element information through a neural network model to obtain the modified element information, wherein the neural network model is trained by using multiple groups of sample data, and each group of data in the multiple groups of sample data comprises: the label of the element information sample and the decoration element information, and the corresponding relation between the element information sample and the decoration element information.

10. The method of claim 9, wherein analyzing the first element information through a neural network model to obtain the modified element information comprises:

analyzing at least one of the following information in the first element information through the neural network model to obtain the modified element information: article information, case information, identification information, original background elements.

11. The method of claim 7, wherein the modifier information comprises: and a decoration element and/or a main body decoration element, wherein the decoration element is used for decorating the target area of the original image, and the main body decoration element is used for decorating the target object in the original image.

12. The method of claim 7, wherein adjusting the original image based on the second element information to obtain a target image of the media file comprises:

and adding the decoration element information to the original image to obtain the target image.

13. The method of any one of claims 1 to 12, wherein the media file is a banner media file of a website page.

14. An image processing method, comprising:

displaying an original image of the media file on an operation interface;

reading first element information of the original image, and acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value;

and displaying a target image of the media file obtained by adjusting the original image based on the second element information in a browsing area of the operation interface.

15. The method of claim 14, wherein reading the first element information of the original image comprises:

reading a structural matrix and a visual matrix of the original image, wherein the original image comprises at least one element, the structural matrix is used for indicating the structure of the at least one element distributed in the original image, and the visual matrix is used for indicating the visual information presented by the at least one element in the original image.

16. The method of claim 14, wherein obtaining second element information corresponding to the first element information comprises:

determining an original background element from the first element information;

17. The method of claim 16, further comprising:

18. The method of claim 14, wherein obtaining second element information corresponding to the first element information comprises:

19. The method of claim 18, further comprising:

20. An image processing method, comprising:

acquiring an original image of a media file, wherein the original image comprises an item to be sold;

acquiring first element information of the original image;

acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value and is used for modifying the article;

21. The method of claim 20, wherein obtaining the first element information of the original image comprises:

22. The method of claim 20, wherein obtaining second element information corresponding to the first element information comprises:

determining an original background element from the first element information;

23. The method of claim 22, wherein adjusting the original image based on the second element information to obtain a target image of the media file comprises:

24. The method of claim 20, wherein obtaining second element information corresponding to the first element information comprises:

25. The method of claim 24, wherein adjusting the original image based on the second element information to obtain a target image of the media file comprises:

26. An image processing method, comprising:

displaying an original image of the media file on an operation interface;

responding to an input operation instruction acted on the operation interface, and acquiring second element information;

reading first element information of the original image corresponding to the second element information, wherein the similarity between the second element information and the first element information is larger than a target threshold value;

27. The method of claim 26, wherein obtaining the second element information comprises:

and acquiring a target feature carried by the input operation instruction, and reading the second element information corresponding to the target feature.

28. The method of claim 27, wherein reading the second elemental information corresponding to the target feature comprises at least one of:

selecting a target background element from at least one element information corresponding to the target feature, wherein the target background element corresponds to an original background element in the first element information, and the second element information comprises the target background element;

and selecting modification element information from at least one element information corresponding to the target feature, wherein the modification element information is matched with the first element information, and the second element information comprises the modification element information.

29. The method of claim 28, wherein the at least one element information is element information of at least one style of image.

30. An image processing apparatus characterized by comprising:

a first acquisition unit for acquiring an original image of a media file;

a second acquisition unit configured to acquire first element information of the original image;

a third obtaining unit configured to obtain second element information corresponding to first element information, where a similarity between the second element information and the first element information is greater than a target threshold;

and the first adjusting unit is used for adjusting the original image based on the second element information to obtain a target image of the media file.

31. An image processing apparatus characterized by comprising:

the first display unit is used for displaying an original image of the media file on the operation interface;

the reading unit is used for reading first element information of the original image and acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value;

and the second display unit is used for displaying a target image of the media file obtained by adjusting the original image based on the second element information in a browsing area of the operation interface.

32. An image processing apparatus characterized by comprising:

the fourth acquisition unit is used for acquiring an original image of the media file, wherein the original image comprises an article to be sold;

a fifth acquiring unit configured to acquire first element information of the original image;

a sixth obtaining unit, configured to obtain second element information corresponding to first element information, where a similarity between the second element information and the first element information is greater than a target threshold, and the sixth obtaining unit is configured to modify the article;

and the second adjusting unit is used for adjusting the original image based on the second element information to obtain a target image of the media file.

33. A storage medium comprising a stored program, wherein the program, when executed, controls an apparatus on which the storage medium is located to perform the steps of:

acquiring an original image of a media file;

acquiring first element information of the original image;

34. A processor, wherein the processor is configured to execute a program, wherein the program executes to perform the following steps:

acquiring an original image of a media file;

acquiring first element information of the original image;

35. A mobile terminal, comprising:

a processor;

the image acquisition device is used for acquiring an original image of the media file; and

the memory is connected with the image acquisition device and used for providing instructions for the processor to process the following processing steps: acquiring first element information of the original image; acquiring second element information corresponding to the first element information, wherein the similarity between the second element information and the first element information is greater than a target threshold value; and adjusting the original image based on the second element information to obtain a target image of the media file.