CN114265948A

CN114265948A - Image pushing method and device

Info

Publication number: CN114265948A
Application number: CN202111610765.3A
Authority: CN
Inventors: 徐立峰; 周维斯; 王勇; 黄勇尤
Original assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Current assignee: Beijing Jingdong Century Trading Co Ltd; Beijing Wodong Tianjun Information Technology Co Ltd
Priority date: 2021-12-27
Filing date: 2021-12-27
Publication date: 2022-04-01
Also published as: WO2023124793A1

Abstract

The embodiment of the disclosure discloses an image pushing method and device. One embodiment of the method comprises: acquiring visual features of an image to be pushed, wherein the visual features comprise design features and target visual features, and the target visual features are obtained by vector embedding; determining the preference of a user to an image to be pushed according to the visual characteristics; and responding to the fact that the preference degree meets the preset condition, and pushing the image to be pushed to a terminal corresponding to the user. The embodiment realizes the effect of improving the accuracy of the image pushing result.

Description

Image pushing method and device

Technical Field

The embodiment of the disclosure relates to the technical field of computers, in particular to an image pushing method and device.

Background

Along with the popularization of the application of the multimedia terminal, the multi-modal creative design plays an increasingly important role in information pushing, and therefore the effects of improving the information pushing efficiency and accuracy and the like by applying the multi-modal creative design are achieved.

The existing implementations mainly include two kinds. One method is post-delivery optimization, and the method mainly comprises the steps of firstly performing multiple groups of creative designs and delivering under gray level flow, and then predicting the optimal delivery proportion of each creative design through a Bayesian algorithm and the like based on gray level effects to perform full delivery. The other method is pre-projection prediction, which is mainly based on multiple groups of creative design images, extracts high-order features of the images in an image Embedding mode, and then performs predictive modeling to determine the creative design image with the highest score for projection.

Disclosure of Invention

The embodiment of the disclosure provides an image pushing method and device.

In a first aspect, an embodiment of the present disclosure provides an image pushing method, including: acquiring visual features of an image to be pushed, wherein the visual features comprise design features and target visual features, and the target visual features are obtained by vector embedding; determining the preference of a user to an image to be pushed according to the visual characteristics; and responding to the fact that the preference degree meets the preset condition, and pushing the image to be pushed to a terminal corresponding to the user.

In a second aspect, an embodiment of the present disclosure provides an image pushing apparatus, including: the visual feature acquisition unit is configured to acquire visual features of an image to be pushed, wherein the visual features comprise design features and target visual features, and the target visual features are obtained by vector embedding; the preference degree determining unit is configured to determine the preference degree of the user to the image to be pushed according to the visual characteristics; and the pushing unit is configured to respond to the fact that the preference degree meets the preset condition, and push the image to be pushed to the terminal corresponding to the user.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: one or more processors; storage means for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method as described in any implementation of the first aspect.

In a fourth aspect, embodiments of the present disclosure provide a computer-readable medium on which a computer program is stored, which computer program, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

According to the image pushing method and device provided by the embodiment of the disclosure, the design features corresponding to the image to be pushed are obtained, the target visual features obtained by vector embedding are used as the visual features, then the preference degree of the user for the image to be pushed is determined according to the visual features, and then the image pushing is carried out according to the preference degree of the user for the image to be pushed, so that the image can be more comprehensively subjected to feature characterization by combining the design features, and the image pushing effect is favorably improved.

Drawings

Other features, objects and advantages of the disclosure will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:

FIG. 1 is an exemplary system architecture diagram in which one embodiment of the present disclosure may be applied;

FIG. 2 is a flow diagram of one embodiment of an image push method according to the present disclosure;

FIG. 3 is a flow diagram of yet another embodiment of an image pushing method according to an embodiment of the present disclosure;

FIG. 4 is a flow diagram of yet another embodiment of an image pushing method according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of an application scenario of an image push method according to an embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of one embodiment of an image pushing device according to the present disclosure;

FIG. 7 is a schematic structural diagram of an electronic device suitable for use in implementing embodiments of the present disclosure.

Detailed Description

The present disclosure is described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.

It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

Fig. 1 shows an exemplary architecture 100 to which an embodiment of an image push method or an image push apparatus of the present disclosure may be applied.

As shown in fig. 1, the system architecture 100 may include

terminal devices

101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the

terminal devices

101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.

The

terminal devices

101, 102, 103 interact with a server 105 via a network 104 to receive or send messages or the like. Various client applications may be installed on the

terminal devices

101, 102, 103. For example, browser-like applications, search-like applications, shopping-like applications, instant messaging tools, social-like applications, image processing-like applications, and the like.

The

terminal apparatuses

101, 102, and 103 may be hardware or software. When the

terminal devices

101, 102, 103 are hardware, they may be various electronic devices including, but not limited to, smart phones, tablet computers, e-book readers, laptop portable computers, desktop computers, and the like. When the

terminal apparatuses

101, 102, 103 are software, they can be installed in the electronic apparatuses listed above. It may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules to provide distributed services) or as a single piece of software or software module. And is not particularly limited herein.

The server 105 may be a server that provides various services, such as a backend server that provides service support for client applications installed on the

terminal devices

101, 102, 103. The server 105 may obtain a visual feature of the image to be pushed, determine a preference degree of the user for the image to be pushed according to the visual feature, and if the preference degree meets a preset condition, may further push the image to be pushed to a terminal (e.g.,

terminal devices

101, 102, 103, etc.) corresponding to the user.

It should be noted that the visual features of the image to be pushed may be directly stored locally in the server 105, and the server 105 may directly extract and process the visual features of the image to be pushed stored locally, in which case, the

terminal devices

101, 102, and 103 and the network 104 may not be present.

It should be noted that the image pushing method provided by the embodiment of the present disclosure is generally executed by the server 105, and accordingly, the image pushing apparatus is generally disposed in the server 105.

It should be further noted that the

terminal devices

101, 102, and 103 may also be installed with image push applications, and the

terminal devices

101, 102, and 103 may also determine, based on the image push applications, a preference degree of a user for an image to be pushed according to a visual feature of the image to be pushed, and push the image to be pushed to a terminal corresponding to the user when the preference degree meets a preset condition. At this time, the image push method may be executed by the

terminal apparatuses

101, 102, 103, and accordingly, the image push apparatus may be provided in the

terminal apparatuses

101, 102, 103. At this point, the exemplary system architecture 100 may not have the server 105 and the network 104.

The server 105 may be hardware or software. When the server 105 is hardware, it may be implemented as a distributed server cluster composed of a plurality of servers, or may be implemented as a single server. When the server 105 is software, it may be implemented as multiple pieces of software or software modules (e.g., multiple pieces of software or software modules used to provide distributed services), or as a single piece of software or software module. And is not particularly limited herein.

It should be understood that the number of terminal devices, networks, and servers in fig. 1 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.

With continued reference to FIG. 2, a flow 200 of one embodiment of an image push method according to the present disclosure is shown. The image pushing method comprises the following steps:

step 201, obtaining the visual characteristics of the image to be pushed.

In the present embodiment, the image to be pushed may be an image of various types and contents. For example, the image to be pushed may be an introduction map of a certain item or the like. As another example, the image to be pushed may be a promotional map of a certain service or activity, and so on. The visual features of the image to be pushed represent the features presented by the image to be pushed in the visual angle. The visual features may include design features and target visual features.

The design feature can be used to represent the feature of the image to be pushed in terms of design. The design features may include various aspects of the design features depending on the actual application scenario or application requirements. For example, including but not limited to: layout, background decoration, style, color system, etc.

The target visual features can be obtained by using vector Embedding (Embedding), and particularly, can be obtained by using various vector Embedding methods for images. In general, the target visual features may be represented using feature vectors.

The visual characteristics of the image to be pushed may be determined based on various methods. For example, the design features may be pre-labeled by the designer. The target visual features can be extracted by using a convolutional neural network and the like. As an example, based on the existing classification model and network structures such as ResNet, a plurality of blocks in a shallow layer may be reserved, and then a full connection layer and a prediction output layer are connected to construct a model and trained, at this time, the output of the full connection layer may be used as a target visual feature

The executing entity of the image pushing method (such as the server 105 shown in fig. 1) may retrieve the visual characteristics of the image to be pushed from a local or other storage device. It should be noted that the visual features of the image to be pushed may be determined by the execution subject of the image pushing method, or may be determined by other electronic devices.

Step 202, determining the preference of the user to the image to be pushed according to the visual characteristics.

In this embodiment, the preference of the user to the image to be pushed may represent the interest level of the user in the image to be pushed. Specifically, the preference may adopt various specific representation modes according to actual application scenarios. For example, the preference may be expressed by data indexes such as click rate and browsing duration.

According to different application scenes, various methods can be adopted to determine the preference degree of the user to the image to be pushed according to the visual characteristics of the image to be pushed. For example, the preference of the user for the image to be pushed can be determined according to the visual features of the image with higher preference of the user in the statistical history period, and then according to the similarity between the visual features obtained by statistics and the visual features of the image to be pushed. Generally, the preference of the user to the image to be pushed is positively correlated with the determined similarity.

For another example, when the preference degree is expressed by using data indexes such as click rate, various data index prediction methods can be used to determine the preference degree of the user to the image to be pushed. By way of example, the click rate of the image to be pushed by the user is determined according to the visual characteristics of the image to be pushed by utilizing various existing click rate prediction models. Generally, the preference of the user to the image to be pushed is positively correlated with the determined click rate.

Step 203, in response to determining that the preference meets the preset condition, pushing the image to be pushed to the terminal corresponding to the user.

In this embodiment, the preset condition can be flexibly set according to the actual application requirement. For example, the preset condition may include that the determined preference degree is not less than a preset threshold value. For another example, the preset condition may include that the determined preference degree corresponds to a sort position not less than the preset position. The ranking position may refer to a position in a ranking formed by ranking the preference degrees corresponding to the preset multiple images to be pushed respectively.

If the preference degree meets the preset condition, the user can be shown to be interested in the image to be pushed. At this time, the image to be pushed can be further pushed to a terminal corresponding to the user (such as a terminal used by the user), so that image pushing based on the design features can be realized, image pushing by only adopting target visual features obtained by vector embedding is avoided, and preference requirements of the user on the design features of the image are ignored.

In some optional implementations of the present embodiment, the design feature of the image to be pushed may be determined by:

step one, analyzing design elements of an image to be pushed.

In this step, in general, the design of the image to be pushed can be embodied by the various design elements it has. The different designs may include a variety of different design elements. For the same design, different partitioning modes may also generate different design elements. For example, design elements include, but are not limited to: color system, font, style, graphics layout, etc.

Specifically, various parsing methods can be adopted to determine the design elements of the image to be pushed. For example, a design document of an image to be pushed may be set in advance by a designer, and design elements may be recorded in the design document. At this time, each design element of the image to be pushed can be analyzed by reading the design document of the image to be pushed.

And step two, identifying the semantics represented by the design elements to obtain an identification result, and determining the obtained identification result as the design feature.

In this step, the semantics represented by the design elements may refer to the intent that the designer desires to be expressed by the design elements. For example, different semantics such as solor or ease may be expressed by different color systems.

By identifying the semantics represented by the design elements, a semantic identification result can be obtained, and the obtained semantic identification result can be determined as a design feature, so that the design intention of the image to be pushed can be converted into an interpretable design feature.

Specifically, the semantics represented by the design elements can be recognized by adopting various different recognition methods according to the actual application scene. For example, the correspondence between design elements and semantics may be pre-labeled by a designer. At this time, the semantics represented by the design elements may be determined by querying the preset correspondence.

For another example, taking color system semantic recognition of the background as an example, the semantic meaning represented by the color system of the background can be determined by the following steps:

step one, counting the color values of all pixel points contained in the background, and screening the first ten color values with the maximum number of corresponding pixel points.

And step two, selecting color values with the saturation not lower than a preset saturation threshold value and the brightness not lower than a preset brightness threshold value from the ten color values to eliminate invalid colors or neutral colors with too low saturation or too low brightness.

And step three, clustering the screened color values to obtain a clustering result.

And fourthly, querying the semantics represented by the clustering clusters in the clustering result by using the preset color value-semantic corresponding relation.

In the prior art, creative design features such as color system emotion tendency, style and the like contained in an image cannot be analyzed only based on the feature extraction and representation mode of the image Embedding, and the creative design features can relatively transfer visual atmosphere and design mental intelligence, so that a pushed image predicted only based on the feature extraction and representation mode of the image Embedding may not accord with the current emotion complaint and mental intelligence transmission of a user. Based on the method, the matching degree between the image and the user can be more comprehensively measured by combining the feature extraction of the Embedding image and the design feature corresponding to the image, so that the accuracy of pushing the image is improved, and the user experience is further improved.

In some optional implementations of this embodiment, the design features may include at least one of: design content characteristics and design performance characteristics.

Wherein designing content features may refer to designing features presented in terms of content. For example, design content features may include, but are not limited to: logo (trademark/Logo), objects present in the image, characters present in the image (such as a paperwork), various controls in the image (such as buttons, etc.), and so forth.

Design performance characteristics may refer to design characteristics in terms of visual performance. For example, design performance characteristics include, but are not limited to: color system, layout, drawing of objects represented in the image, background decoration, style, and the like.

By analyzing and identifying the design characteristics of the image to be pushed from the aspects of content and expression, the design characteristics of the image to be pushed can be visualized, and the design characteristics of the image to be pushed can be more comprehensively and conveniently represented in the mode.

Alternatively, the design content characteristics may be determined by:

step one, analyzing design content elements of an image to be pushed based on semantic segmentation.

In this step, various existing Semantic Segmentation (Semantic Segmentation) methods can be adopted to identify each design content element from the image to be pushed.

In some cases, the design content element set may be preset according to a requirement, and at this time, the design content elements belonging to the preset design content element set and possessed by the image to be pushed may be identified from the image to be pushed.

For example, each design content element that it has can be identified from the image to be pushed by using a pre-trained design content element identification model implemented based on semantic segmentation. As an example, an object region, a character region, a Logo region, a space region, and the like in the image to be pushed may be identified.

And secondly, identifying the semantics represented by the design content elements to obtain an identification result, and determining the obtained identification result as the design content features.

The semantic identification of this step can refer to the related description above, and is not described herein again.

Each design element can be accurately identified by utilizing semantic segmentation, so that the design characteristics of the image to be pushed can be more accurately represented.

Alternatively, the design performance characteristics may be determined by:

analyzing a design file of an image to be pushed to obtain a design expression element of the image to be pushed.

In this step, the design file of the image to be pushed may refer to various data formed in the design process of the image to be pushed. For example, if the image to be pushed is produced by using a certain design tool or application, a file formed by using the design tool or application may be used as a design file of the image to be pushed.

Different parsing methods can be adopted for different types of design files. For example, the number of layers, the type of layers, the size of layers, and the like corresponding to the design file may be read first, and then the content of each layer may be specifically analyzed to determine the design element of the image to be pushed.

And step two, identifying the semantics expressed by the design expression elements to obtain an identification result, and determining the obtained identification result as the design expression characteristic.

The original design semantics can be known through analysis and identification of the design file of the image to be pushed, so that the design features can be more accurately expressed.

In some cases, the target visual feature may be obtained by processing the image to be pushed, or may be obtained by processing an image template corresponding to the image to be pushed by using vector embedding. For example, a convolutional neural network is used to extract a feature vector from an image template corresponding to the image to be pushed. The image template may refer to a design template of an image to be pushed, may include each design element, and may not include specific content of each design element.

At this time, after it is determined that the preference of the user to the image to be pushed meets the preset condition, the content of the design element indicated by the design feature of the image to be pushed can be supplemented in the design template, the supplemented image is used as the image to be pushed, and the obtained image to be pushed is pushed to the user.

In the existing image push based on post-delivery tuning, because enough prior knowledge may be lacked in each creative design delivery, a gray scale experiment is usually required to be performed before delivery tuning. However, a flow equalization method is usually adopted during the gray scale experiment, so that the optimization of the flow use efficiency cannot be realized, and the situation that a poor creative design occupies a large flow proportion is easy to occur.

The method provided by the embodiment of the disclosure determines the preference degree of the user to the image to be pushed by combining the design features of the image to be pushed and the target visual features obtained by utilizing the Embedding of the image, and then carries out image pushing based on the preference degree, so that the processes that the gray scale needs to be carried out for realizing the releasing after releasing and the like can be avoided, and on the basis of utilizing the high-order features of the image, the preference degree prediction is further carried out by further combining creative design semantics such as color system emotion tendency, style and the like expressed by the design features of the image, which is beneficial to improving the accuracy of the image pushing result, so that the pushed image is more matched with the requirements of the user in the aspect of design.

With further reference to fig. 3, a flow 300 of yet another embodiment of an image push method is shown. The flow 300 of the image pushing method includes the following steps:

step 301, obtaining the visual characteristics of the image to be pushed.

Step 302, obtaining the associated features of the image to be pushed.

In the present embodiment, the associated feature may refer to various features associated with an image to be pushed in addition to the visual feature. The association characteristic may include at least one of: attribute features, text features.

Wherein the text feature may represent a feature of a character in the image to be pushed. The attribute characteristics may include at least one of: attribute characteristics of the user and attribute characteristics of an object described by the image to be pushed. The attribute characteristics of the user may include inherent attributes, behavioral attributes, and the like of the user. The objects described by the image to be pushed may be various types of objects. Different types of objects may have different attribute characteristics.

For example, if the image to be pushed is an introduction diagram of an item, the object described by the image to be pushed may be the item. At this time, the attribute feature of the object is the attribute feature of the article (such as composition, production place, shelf life, etc.). For another example, if the image to be pushed is a promotion diagram of a certain service, the object described by the image to be pushed may be the service. At this time, the attribute feature of the object is the attribute feature of the service (such as start time, end time, location, etc.).

The associated characteristics of the image to be pushed can be determined in various different ways. For example, for attribute features, the determination may be made by collecting relevant attribute data. For another example, for the text feature, a Character in the image to be pushed may be extracted based on a method such as OCR (Optical Character Recognition), and then the corresponding text feature may be extracted by a method such as NLP (Natural Language Processing).

For example, a Long Short-Term Memory network (LSTM) can be constructed and trained, and finally the output Embelling result of the fully-connected layer can be used as a case feature.

The executing agent may obtain the associated features of the image to be pushed from a local or other data source. It should be noted that the associated features of the image to be pushed may be determined by the execution subject, or may be determined by other electronic devices. The visual features and the associated features of the image to be pushed may be determined by the same electronic device or may be determined by different electronic devices.

Step 303, determining the preference of the user to the image to be pushed according to the visual characteristics and the associated characteristics.

In this embodiment, the visual features and the associated features of the image to be pushed may be integrated to determine the preference of the user to the image to be pushed. Specifically, various preference determination methods can be adopted according to actual application requirements.

For example, a first preference of the user for the image to be pushed may be determined according to the visual feature of the image to be pushed, and then a second preference of the user for the image to be pushed may be determined according to the associated feature of the image to be pushed. And then, fusing the first preference and the second preference in a weighting sum mode and the like, and determining a fusion result as the preference of the user to the image to be pushed.

For another example, the preference degree of the user to the image to be pushed can be determined according to the visual features and the associated features of the image to be pushed by utilizing various neural network models based on deep learning. As an example, the preference degree determining model may be trained in advance, the visual features and the associated features of the image to be pushed are used as inputs, and the preference degree of the user for the image to be pushed is used as an output, so that the preference degree of the user for the image to be pushed is obtained by using the preference degree determining model.

And 304, in response to the fact that the preference degree meets the preset condition, pushing the image to be pushed to the terminal corresponding to the user.

In some cases, the target visual feature and the text feature of the image to be pushed can be realized based on a neural network model, and the preference prediction and the pushing prediction can be realized based on the neural network model. Specifically, training data may be extracted by collecting historical behavior records of the user, or the like, to train the relevant neural network model. Each different neural network model can be trained independently, and multi-mode data fusion, poor performance and the like can be carried out in the training process to realize multi-task training so as to improve the training effect of each neural network model. For example, a neural network model based on vector embedding and feature vector extraction may be trained, in which case, the extracted feature vectors may represent target visual features, and the neural network model extracting text features may be trained, and then these models may be used as pre-training models, joint training image push prediction models or preference prediction models, and so on.

The method provided by the above embodiment of the present disclosure performs image pushing by combining the feature data of three different modalities, namely, the visual feature, the text feature and the attribute feature of the image to be pushed, and generally, the attribute of the object described by the image, the user attribute, the visual feature (such as an object display diagram presented in the image, etc.), a document, etc. are several aspects that are relatively easy to attract users, so that various factors affecting the pushing effect in the image pushing scene can be considered comprehensively by using the feature data of the three different modalities, namely, the visual feature, the text feature and the attribute feature, thereby contributing to improving the image pushing effect.

With further reference to fig. 4, a flow 400 of yet another embodiment of an image push method is shown. The flow 400 of the image pushing method includes the following steps:

step 401, obtaining the visual characteristics of the image to be pushed.

Step 402, determining the preference of the user to the image to be pushed according to the visual characteristics.

And step 403, in response to determining that the preference meets the preset condition, pushing the image to be pushed to the terminal corresponding to the user.

And step 404, acquiring operation information of the user aiming at the image to be pushed.

In this embodiment, the operation information may be used to indicate an operation performed by the user. The operation indicated by the operation information may refer to various operations performed by the user after receiving the image to be pushed. For example, the user's operations may include: click/not click, browse/not browse, favorites/not favorites, comment/not comment, and so on.

Generally, the terminal corresponding to the user may collect operation information of the user for the image to be pushed, and then acquire the operation information of the user for the image to be pushed from the terminal of the user object.

And step 405, constructing a knowledge graph according to the operation information.

In this embodiment, the knowledge-graph may be used to record user preference information for visual features. The preference information may indicate a preference of the user. Generally, the user's preference for pushed images can be known from the user's operation information, and then recorded to construct a knowledge graph. The content recorded by the knowledge graph can be flexibly set according to the actual application requirement.

As an example, if a user clicks on an image to be pushed for presenting an item, the user identification, the item identification, the design element, the semantics of the design element, and the design intent may be recorded. The design intention may refer to a design intention of a designer when designing an image to be pushed. At this time, the structure of the knowledge-graph may be as shown in table 1:

as shown in table 1, a knowledge graph may be constructed to record correspondence between items, users, design elements, design semantics, and scene intents. Taking saturation as an example, the semantics of saturation may specifically include low saturation, medium saturation, and high saturation, and the sequential changes in these three saturations may correspond to a change in scene intent from young to mature. Therefore, if it is determined that the user prefers a low saturation design semantic based on the operator information of the user, it is possible to record in the knowledge map to indicate that the user likes a young design style.

It should be noted that the construction of the knowledge graph is usually a continuous iterative and settling process, so that the knowledge graph can be continuously updated according to the continuous image pushing result in practical application.

TABLE 1

The conventional image pushing based on post-projection optimization cannot analyze creative design, so that reusable knowledge cannot be deposited in each creative design prediction. Because the image Embedding itself is a high-order feature and usually participates in calculation in a vector form, the image Embedding based on image Embedding cannot know which factors or features contribute to the screened better creative design, and further cannot design according to the creative design guided by the screened creative design.

In view of these situations, the method provided by the above embodiment of the present disclosure may implement extraction of interpretable low-order design features of an image to be pushed by using design file analysis or analysis based on semantic segmentation and design semantic recognition, so as to solve the problem of unexplainable caused by image Embedding, and may perform knowledge recording, precipitation and transformation according to the pushing effect of each time by constructing a knowledge graph, so as to further assist in various application scenarios such as subsequent image pushing, image searching and the like for a user by using the constructed knowledge graph.

Referring now to fig. 5, fig. 5 is an exemplary application scenario 500 of the image push method according to the present embodiment. In the application scenario of fig. 5, for each design template of a specific article, corresponding design features 5011 may be determined based on algorithms such as design file parsing and semantic segmentation, and the corresponding relationship 504 between the design template and the corresponding design features may be recorded. Meanwhile, a feature vector of the design template can be extracted by using a convolutional neural network to serve as the target visual feature 5012, so that the visual feature 501 consisting of the design feature 5011 and the target visual feature 5012 is obtained.

In addition, the text feature 502 corresponding to the design template may be extracted based on technologies such as OCR and NLP, and the attribute feature 503 composed of the user attribute feature 5031 and the item attribute feature 5032 may be obtained in advance based on statistical analysis.

Then, the visual features 501, the text features 502, and the attribute features 503 may be input to a click rate prediction model 505 trained in advance, so as to obtain a click rate 506 corresponding to the design template. And then, according to the click rate corresponding to each design template, selecting the design template with a larger click rate, adding the design characteristics corresponding to the design template into the design template to form an image and pushing the image to a user.

With further reference to fig. 6, as an implementation of the method shown in the above figures, the present disclosure provides an embodiment of an image pushing device, which corresponds to the embodiment of the method shown in fig. 2, and which is particularly applicable in various electronic devices.

As shown in fig. 6, the image push apparatus 600 provided by the present embodiment includes a visual feature acquisition unit 601, a preference degree determination unit 602, and a push unit 603. The visual feature obtaining unit 601 is configured to obtain visual features of an image to be pushed, where the visual features include design features and target visual features, and the target visual features are obtained by vector embedding; the preference determining unit 602 is configured to determine a preference of the user for the image to be pushed according to the visual characteristics; the pushing unit 603 is configured to, in response to determining that the preference meets the preset condition, push the image to be pushed to the terminal corresponding to the user.

In the present embodiment, in the image pushing apparatus 600: the specific processing of the visual characteristic obtaining unit 601, the preference determining unit 602, and the pushing unit 603 and the technical effects thereof can refer to the related descriptions of step 201, step 202, and step 203 in the corresponding embodiment of fig. 2, which are not described herein again.

In some alternative implementations of this embodiment, the design features are determined by: analyzing design elements of the image to be pushed; the method includes identifying semantics represented by the design elements, obtaining an identification result, and determining the obtained identification result as a design feature.

In some optional implementations of the embodiment, the design feature includes at least one of: design content characteristics and design performance characteristics.

In some optional implementations of this embodiment, the design content characteristics are determined by: analyzing design content elements of the image to be pushed based on semantic segmentation; identifying semantics represented by the design content elements, obtaining an identification result, and determining the obtained identification result as a design content feature.

In some alternative implementations of this embodiment, the design performance characteristic is determined by: analyzing a design file of the image to be pushed to obtain a design expression element of the image to be pushed; the method includes identifying semantics represented by the design presentation elements, obtaining an identification result, and determining the obtained identification result as a design presentation feature.

In some optional implementations of the present embodiment, the image pushing apparatus further includes: an associated feature obtaining unit (not shown in the figure) configured to obtain an associated feature of the image to be pushed, wherein the associated feature includes at least one of the following: attribute features and text features, wherein the attribute features comprise at least one of the following: attribute characteristics of a user and attribute characteristics of an object described by an image to be pushed; and the preference determining unit 602 is further configured to determine the preference of the user to the image to be pushed according to the visual feature and the associated feature.

In some optional implementations of the present embodiment, the image pushing apparatus further includes: an operation information acquisition unit (not shown in the figure) configured to acquire operation information of a user for an image to be pushed; and the construction unit is configured to construct a knowledge graph according to the operation information, wherein the knowledge graph is used for recording preference information of the user for the visual characteristics.

According to the device provided by the embodiment of the disclosure, the visual characteristics of the image to be pushed are obtained through the visual characteristic obtaining unit, wherein the visual characteristics comprise design characteristics and target visual characteristics, and the target visual characteristics are obtained by vector embedding; the preference degree determining unit determines the preference degree of the user to the image to be pushed according to the visual characteristics; the pushing unit is used for pushing the image to be pushed to the terminal corresponding to the user in response to the fact that the determined preference degree meets the preset condition, on the basis that the high-order features of the image are utilized, the preference degree prediction can be further carried out in combination with creative design semantics such as color system emotion tendency and style expressed by the design features of the image, the accuracy of an image pushing result is improved, and the pushed image is enabled to be more matched with the requirements of the user in the aspect of design.

Referring now to FIG. 7, a block diagram of an electronic device (e.g., the server of FIG. 1) 700 suitable for use in implementing embodiments of the present disclosure is shown. The server shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 7, electronic device 700 may include a processing means (e.g., central processing unit, graphics processor, etc.) 701 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)702 or a program loaded from storage 708 into a Random Access Memory (RAM) 703. In the RAM703, various programs and data necessary for the operation of the electronic apparatus 700 are also stored. The processing device 701, the ROM 702, and the RAM703 are connected to each other by a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.

Generally, the following devices may be connected to the I/O interface 705: input devices 706 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 707 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 708 including, for example, magnetic tape, hard disk, etc.; and a communication device 709. The communication means 709 may allow the electronic device 700 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 illustrates an electronic device 700 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided. Each block shown in fig. 7 may represent one device or may represent multiple devices as desired.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via the communication means 709, or may be installed from the storage means 708, or may be installed from the ROM 702. The computer program, when executed by the processing device 701, performs the above-described functions defined in the methods of embodiments of the present disclosure.

It should be noted that the computer readable medium described in the embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In embodiments of the present disclosure, however, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: the method comprises the steps of obtaining visual features of an image to be pushed, wherein the visual features comprise design features and target visual features, and the target visual features are embedded by utilizing vectors to obtain feature vectors; determining the preference of a user to an image to be pushed according to the visual characteristics; and responding to the fact that the preference degree meets the preset condition, and pushing the image to be pushed to a terminal corresponding to the user.

Computer program code for carrying out operations for embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present disclosure may be implemented by software or hardware. The described units may also be provided in a processor, and may be described as: a processor includes a visual feature acquisition unit, a preference determination unit, and a push unit. The names of the units do not form a limitation on the units themselves in some cases, for example, the visual feature acquisition unit may also be described as a unit for acquiring visual features of an image to be pushed, wherein the visual features include design features and target visual features, and the target visual features are obtained by vector embedding.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above-mentioned features, but also encompasses other embodiments in which any combination of the above-mentioned features or their equivalents is made without departing from the inventive concept as defined above. For example, the above features and (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure are mutually replaced to form the technical solution.

Claims

1. An image pushing method, comprising:

the method comprises the steps of obtaining visual features of an image to be pushed, wherein the visual features comprise design features and target visual features, and the target visual features are obtained by vector embedding;

determining the preference degree of a user for the image to be pushed according to the visual features;

and pushing the image to be pushed to a terminal corresponding to the user in response to the fact that the preference degree meets the preset condition.

2. The method of claim 1, wherein the design feature is determined by:

analyzing the design elements of the image to be pushed;

identifying semantics represented by the design elements, obtaining an identification result, and determining the obtained identification result as a design feature.

3. The method of claim 2, wherein the design feature comprises at least one of: design content characteristics and design performance characteristics.

4. The method of claim 3, wherein the design content characteristics are determined by:

analyzing design content elements of the image to be pushed based on semantic segmentation;

identifying semantics represented by the design content elements, obtaining an identification result, and determining the obtained identification result as a design content feature.

5. The method of claim 3, wherein the design performance characteristic is determined by:

analyzing the design file of the image to be pushed to obtain a design expression element of the image to be pushed;

identifying semantics represented by the design expression elements, obtaining an identification result, and determining the obtained identification result as a design expression characteristic.

6. The method of claim 1, wherein the method further comprises:

acquiring the associated features of the image to be pushed, wherein the associated features comprise at least one of the following items: attribute features and text features, wherein the attribute features comprise at least one of the following: the attribute characteristics of the user and the attribute characteristics of the object described by the image to be pushed; and

the determining the preference degree of the user for the image to be pushed according to the visual features comprises the following steps:

and determining the preference degree of the user for the image to be pushed according to the visual features and the associated features.

7. The method according to one of claims 1 to 6, wherein after the image to be pushed is pushed to the terminal corresponding to the user, the method further comprises:

acquiring operation information of the user for the image to be pushed;

and constructing a knowledge graph according to the operation information, wherein the knowledge graph is used for recording preference information of the user for visual features.

8. An image pushing apparatus, wherein the apparatus comprises:

the visual feature acquisition unit is configured to acquire visual features of an image to be pushed, wherein the visual features comprise design features and target visual features, and the target visual features are obtained by vector embedding;

the preference degree determining unit is configured to determine the preference degree of the user on the image to be pushed according to the visual characteristics;

the pushing unit is configured to respond to the fact that the preference degree meets the preset condition, and push the image to be pushed to a terminal corresponding to the user.

9. The apparatus of claim 8, wherein the design feature is determined by:

analyzing the design elements of the image to be pushed;

10. The apparatus of claim 9, wherein the design feature comprises at least one of: design content characteristics and design performance characteristics.

11. The apparatus of claim 10, wherein the design content characteristics are determined by:

12. The apparatus of claim 10, wherein the design performance characteristic is determined by:

13. The apparatus of claim 8, wherein the apparatus further comprises:

an associated feature acquiring unit configured to acquire an associated feature of the image to be pushed, wherein the associated feature includes at least one of: attribute features and text features, wherein the attribute features comprise at least one of the following: the attribute characteristics of the user and the attribute characteristics of the object described by the image to be pushed; and

the preference degree determining unit is further configured to determine the preference degree of the user for the image to be pushed according to the visual feature and the associated feature.

14. The apparatus according to one of claims 8-13, wherein the apparatus further comprises:

an operation information acquisition unit configured to acquire operation information of the user for the image to be pushed after the image to be pushed is pushed to a terminal corresponding to the user;

a construction unit configured to construct a knowledge graph according to the operation information, wherein the knowledge graph is used for recording preference information of the user for visual features.

15. An electronic device/terminal/server comprising:

one or more processors;

a storage device having one or more programs stored thereon;

when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-7.

16. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.