CN112837108A

CN112837108A - Information processing method and device and electronic equipment

Info

Publication number: CN112837108A
Application number: CN201911163055.3A
Authority: CN
Inventors: 张晟; 陈彤; 吴春松; 鲍军; 岳阳
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-11-25
Filing date: 2019-11-25
Publication date: 2021-05-25

Abstract

The embodiment of the application discloses an information processing method, an information processing device and electronic equipment, wherein the method comprises the following steps: acquiring at least one neural network model for carrying out feature extraction on the commodity object picture; inputting a target commodity object picture into the neural network model to obtain an output result of an output layer of the neural network model and an activation value of at least one intermediate layer; the activation value of the middle layer represents feature information extracted from the commodity object picture by the middle layer of the neural network model in the process of obtaining an output result, or feature information obtained by performing weighted summation and nonlinear transformation on neurons in the upper layer; and determining characteristic values on multiple dimensions according to the output result of the output layer and the activation value of the intermediate layer, and generating a characteristic vector of the commodity object picture. By the embodiment of the application, the quality of the recommendation information can be improved, and the resource waste is reduced.

Description

Information processing method and device and electronic equipment

Technical Field

The present application relates to the field of information processing technologies, and in particular, to an information processing method and apparatus, and an electronic device.

Background

In a commodity object information service system, recommendation information of a commodity object is often provided to a user in various ways. For example, in the client top page, recommended commodity object information may be provided to the user in the form of a message stream or the like by "guessing you like" block. The reference information in the specific recommendation algorithm may be various, for example, the system may locate the style preference of the buyer according to the usual browsing habit of the consumer and information of browsing records, collected products, and the like, and recommend a similar commodity object to the consumer according to the style preference, and may display the commodity object through a picture of the commodity object. In addition, the system can also position the consumption level of the crowd, and can determine the proper crowd according to the attribute, price and other information of the commodity object during recommendation so as to achieve the expected effect of the user; furthermore, information such as popularity (including click rate, collection rate, purchase rate and the like) of the commodity objects can be integrated, and commodity objects with higher quality can be recommended to the consumer user.

In summary, in the prior art, the method for providing recommendation information mainly selects a specific recommended commodity object for the user according to the category, price, popularity and other information of the commodity object. However, in a specific implementation, the actual click or purchase rate of the recommendation information by the user may not be high, and most of the recommendation information may be ignored due to not really meeting the user's preference, so that various resources such as the occupied computing and network resources are wasted.

Therefore, how to further improve the quality of the recommendation information and reduce the resource waste becomes a technical problem to be solved by those skilled in the art.

Disclosure of Invention

The application provides an information processing method, an information processing device and electronic equipment, which can improve the quality of recommended information and reduce resource waste.

The application provides the following scheme:

an information processing method comprising:

acquiring at least one neural network model for carrying out feature extraction on the commodity object picture;

inputting a target commodity object picture into the neural network model to obtain an output result of an output layer of the neural network model and an activation value of at least one intermediate layer; the activation value of the middle layer represents feature information extracted from the commodity object picture by the middle layer of the neural network model in the process of obtaining an output result, or feature information obtained by performing weighted summation and nonlinear transformation on neurons in the upper layer;

and determining characteristic values on multiple dimensions according to the output result of the output layer and the activation value of the intermediate layer, and generating a characteristic vector of the commodity object picture.

A method of determining user characteristic information, comprising:

determining picture information of a plurality of commodity objects associated with historical behavior information of a user;

respectively inputting pictures of a plurality of commodity objects into the neural network model to obtain an output result of an output layer of the neural network model and an activation value of at least one intermediate layer; the activation value of the middle layer represents feature information extracted from the commodity object picture by the middle layer of the neural network model in the process of obtaining an output result, or feature information obtained by performing weighted summation and nonlinear transformation on neurons in the upper layer;

and determining characteristic values on multiple dimensions by counting the output results of the output layers respectively corresponding to the pictures of the commodity objects and the activation values of the middle layer so as to generate the characteristic vector of the user.

A commodity object recommendation method includes:

determining a target recommendation user of the commodity object and a source database of the commodity object recommendation information;

acquiring feature vector information of the user and feature vector information of a plurality of commodity objects in the source database, wherein the feature vectors of the commodity objects comprise feature values of pictures of the commodity objects in a plurality of dimensions, and the feature values comprise: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the intermediate layer is the characteristic information extracted from the picture of the commodity object by the intermediate layer of the neural network model in the process of obtaining the output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on the neuron on the upper layer;

and inputting the feature vector information of the user and the feature vector information of the plurality of commodity objects into a recommendation algorithm to predict click rate information obtained after recommending the commodity objects to the user.

A commodity object recommendation method includes:

determining picture information of a commodity object to be recommended;

acquiring feature vector information of the commodity object, wherein the feature vector of the commodity object comprises feature values of the picture in multiple dimensions, and the feature values comprise: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

acquiring feature vector information of a plurality of users;

and inputting the characteristic vector information of the commodity object and the characteristic vector information of the users into a recommendation algorithm to predict click rate information obtained after the commodity object to be recommended is recommended to the users.

A method of providing a merchandise object, comprising:

in the process of providing similar commodity object information according to a target commodity object, acquiring feature vector information of the target commodity object and feature vector information of a plurality of commodity objects in a source database; the feature vector of the commodity object comprises feature values of pictures of the commodity object in multiple dimensions, wherein the feature values comprise: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

and comparing the similarity of the characteristic vector information of the target commodity object with the characteristic vector information of a plurality of commodity objects in a source database to provide commodity object information with the similarity meeting the condition of the specified commodity object.

A method of providing merchandise object information, comprising:

receiving an access request to a target page, wherein the target page is associated with a plurality of resource positions and is used for displaying information of a plurality of commodity objects, and the commodity objects are associated with a plurality of different pictures;

acquiring feature vector information corresponding to the multiple different pictures and feature vector information of the visitor user; wherein, the feature vector corresponding to the picture comprises feature values of the picture in multiple dimensions, and the feature values comprise: after the pictures are input into at least one neural network model for carrying out feature extraction on the commodity object pictures, outputting the output result of the layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

respectively inputting the feature vector information of the visitor user and the feature vector information corresponding to the multiple pictures of the commodity object into a prediction algorithm so as to predict click rate information respectively obtained after the multiple pictures of the commodity object are displayed to the user;

and according to the prediction result, displaying the picture with the click rate meeting the condition as a representative picture of the corresponding commodity object in the resource position corresponding to the target page.

An information recommendation method, comprising:

determining a picture associated with the target commodity object;

determining display scene category information corresponding to the commodity object in the picture by performing feature analysis on the picture;

determining a user set, and acquiring user characteristic information corresponding to a plurality of users in the user set respectively, wherein the user characteristic information comprises display scene category information which is interesting to the users;

determining a target user according to the display scene category information corresponding to the picture and the display scene category information interested by the user;

and recommending the picture to a client associated with the target user.

An information recommendation method, comprising:

determining a video associated with the target commodity object;

extracting at least one frame of image from the video, and determining display scene category information corresponding to the commodity object in the video by performing feature analysis on the image;

determining a target user according to the display scene category information corresponding to the video and the display scene category information which is interested by the user;

and recommending the video to the client associated with the target user.

An information processing apparatus comprising:

the model acquisition unit is used for acquiring at least one neural network model for carrying out feature extraction on the commodity object picture;

the analysis unit is used for inputting the target commodity object picture into the neural network model to obtain an output result of an output layer of the neural network model and an activation value of at least one intermediate layer; the activation value of the middle layer represents feature information extracted from the commodity object picture by the middle layer of the neural network model in the process of obtaining an output result, or feature information obtained by performing weighted summation and nonlinear transformation on neurons in the upper layer;

and the picture characteristic vector generating unit is used for determining characteristic values on a plurality of dimensions according to the output result of the output layer and the activation value of the intermediate layer and generating the characteristic vector of the commodity object picture.

An apparatus for determining user characteristic information, comprising:

the picture information determining unit is used for determining picture information of a plurality of commodity objects related to historical behavior information of the user;

the analysis unit is used for respectively inputting the pictures of the plurality of commodity objects into the neural network model to obtain the output result of the output layer of the neural network model and the activation value of at least one intermediate layer; the activation value of the middle layer represents feature information extracted from the commodity object picture by the middle layer of the neural network model in the process of obtaining an output result, or feature information obtained by performing weighted summation and nonlinear transformation on neurons in the upper layer;

and the user characteristic vector generating unit is used for determining characteristic values on a plurality of dimensions by counting the output results of the output layers respectively corresponding to the pictures of the commodity objects and the activation values of the middle layer so as to generate the characteristic vector of the user.

A commodity object information recommending apparatus comprising:

the information determining unit is used for determining a target recommending user of the commodity object and a source database of the commodity object recommending information;

a feature vector obtaining unit, configured to obtain feature vector information of the user and feature vector information of a plurality of commodity objects in the source database, where the feature vector of a commodity object includes feature values of pictures of the commodity object in multiple dimensions, and the feature values include: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the intermediate layer is the characteristic information extracted from the picture of the commodity object by the intermediate layer of the neural network model in the process of obtaining the output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on the neuron on the upper layer;

and the prediction unit is used for inputting the characteristic vector information of the user and the characteristic vector information of the plurality of commodity objects into a recommendation algorithm so as to predict click rate information obtained after the commodity objects are recommended to the user.

A commodity object recommending apparatus, comprising:

the picture information determining unit is used for determining the picture information of the commodity object to be recommended;

a commodity feature vector obtaining unit, configured to obtain feature vector information of the commodity object, where the feature vector of the commodity object includes feature values of the picture in multiple dimensions, and the feature values include: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

a user feature vector acquisition unit having feature vector information on acquiring a plurality of users;

and the prediction unit is used for inputting the characteristic vector information of the commodity object and the characteristic vector information of the users into a recommendation algorithm so as to predict click rate information obtained after the commodity object to be recommended is recommended to the users.

A merchandise object providing apparatus comprising:

the characteristic vector acquisition unit is used for acquiring characteristic vector information of a target commodity object and characteristic vector information of a plurality of commodity objects in a source database in the process of providing similar commodity object information according to the target commodity object; the feature vector of the commodity object comprises feature values of pictures of the commodity object in multiple dimensions, wherein the feature values comprise: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

and the similar commodity object information providing unit is used for comparing the similarity of the characteristic vector information of the target commodity object with the characteristic vector information of a plurality of commodity objects in a source database, and providing the commodity object information with the similarity meeting the condition of the specified commodity object.

An apparatus for providing commodity object information, comprising:

the system comprises an access request receiving unit, a processing unit and a display unit, wherein the access request receiving unit is used for receiving an access request to a target page, the target page is associated with a plurality of resource positions and is used for displaying information of a plurality of commodity objects, and the commodity objects are associated with a plurality of different pictures;

the characteristic vector acquisition unit is used for acquiring characteristic vector information corresponding to the different pictures and characteristic vector information of the visitor user; wherein, the feature vector corresponding to the picture comprises feature values of the picture in multiple dimensions, and the feature values comprise: after the pictures are input into at least one neural network model for carrying out feature extraction on the commodity object pictures, outputting the output result of the layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

the prediction unit is used for respectively inputting the feature vector information of the visitor user and the feature vector information corresponding to the multiple pictures of the commodity object into a prediction algorithm so as to predict click rate information respectively obtained after the multiple pictures of the commodity object are displayed to the user;

and the representative picture determining unit is used for displaying the picture with the click rate meeting the conditions as the representative picture of the corresponding commodity object in the resource position corresponding to the target page according to the prediction result.

An information recommendation apparatus comprising:

the picture determining unit is used for determining a picture associated with the target commodity object;

the display scene determining unit is used for determining display scene category information corresponding to the commodity object in the picture by performing characteristic analysis on the picture;

the system comprises a user characteristic information acquisition unit, a display unit and a display unit, wherein the user characteristic information acquisition unit is used for determining a user set and acquiring user characteristic information corresponding to a plurality of users in the user set respectively, and the user characteristic information comprises display scene category information which is interesting to the users;

the target user determining unit is used for determining a target user according to the display scene category information corresponding to the picture and the display scene category information which is interested by the user;

and the picture recommending unit is used for recommending the picture to the client associated with the target user.

An information recommendation apparatus comprising:

the video determining unit is used for determining a video associated with the target commodity object;

the display scene determining unit is used for extracting at least one frame of image from the video and determining display scene category information corresponding to commodity objects in the video by performing feature analysis on the image;

the target user determining unit is used for determining a target user according to the display scene category information corresponding to the video and the display scene category information which is interested by the user;

and the video recommending unit is used for recommending the video to the client associated with the target user.

An electronic device, comprising:

one or more processors; and

a memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform a method as described in the preceding items.

According to the specific embodiments provided herein, the present application discloses the following technical effects:

through the embodiment of the application, the characteristic extraction can be carried out on the commodity object picture through the neural network model, the output result of the model output layer can be used as the characteristic value on the specific dimension in the picture characteristic vector, and the activation value of the model intermediate layer can be used as the characteristic value on more dimensions in the characteristic vector. In this way, a high-dimensional feature vector can be generated for the commodity object picture, wherein the feature vector can include not only the feature value in the dimension (output result of the output layer) which can be understood and defined by human beings, but also the feature value in the dimension (activation value of the middle layer) which can not be understood or accurately defined by human beings, so that the feature vector can express the feature of the picture more completely. And furthermore, in the scenes of commodity object information recommendation and the like, more accurate recommendation results can be obtained, the click rate of recommendation information is improved, and the resource waste is reduced.

Of course, it is not necessary for any product to achieve all of the above-described advantages at the same time for the practice of the present application.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a schematic diagram of a system architecture provided by an embodiment of the present application;

FIG. 2 is a flow chart of a first method provided by an embodiment of the present application;

FIG. 3 is a schematic diagram of a neural network architecture provided by an embodiment of the present application;

FIG. 4 is a flow chart of a second method provided by embodiments of the present application;

FIG. 5 is a flow chart of a third method provided by embodiments of the present application;

FIG. 6 is a flow chart of a fourth method provided by embodiments of the present application;

FIG. 7 is a flow chart of a fifth method provided by embodiments of the present application;

FIG. 8 is a flow chart of a sixth method provided by embodiments of the present application;

fig. 9 is a flow chart of a seventh method provided by an embodiment of the present application;

fig. 10 is a flow chart of an eighth method provided by an embodiment of the present application;

FIG. 11 is a schematic diagram of a first apparatus provided by an embodiment of the present application;

FIG. 12 is a schematic diagram of a second apparatus provided by an embodiment of the present application;

FIG. 13 is a schematic diagram of a third apparatus provided by an embodiment of the present application;

FIG. 14 is a schematic diagram of a fourth apparatus provided by an embodiment of the present application;

FIG. 15 is a schematic diagram of a fifth apparatus provided by an embodiment of the present application;

FIG. 16 is a schematic view of a sixth apparatus provided by an embodiment of the present application;

FIG. 17 is a schematic diagram of a seventh apparatus provided by an embodiment of the present application;

FIG. 18 is a schematic diagram of an eighth apparatus provided by an embodiment of the present application;

fig. 19 is a schematic diagram of an electronic device provided in an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments that can be derived from the embodiments given herein by a person of ordinary skill in the art are intended to be within the scope of the present disclosure.

It should be noted that, in the process of implementing the present application, the inventor of the present application finds that a picture of a commodity object generally has some features. For example, there may be respective dominant hue information in a picture; further, for a commodity object such as clothing, there may be a difference in a shooting scene, "street shooting" may be performed outdoors, "booth shooting" may be performed in an indoor studio, "a character model may be displayed in a use state," a detail display may be performed without a model, "a display range, a body posture, an expression, and the like of the character model may be different in a case where the character model is displayed. Different features convey different degrees of tonality to the user, and different users may have different degrees of interest in the various features. Therefore, when some recommended commodity object information is provided to the user, whether the recommended commodity object can be operated by the user such as clicking or the like is closely related to whether the picture of the recommended commodity object matches the interest point of the user in addition to the basic information such as the category and the brand of the commodity object. For example, a user prefers the color of a card for a commodity object such as a windcheat, and in the process of browsing information of the commodity object, a picture of the commodity object clicked by the user mostly has the following characteristics: the pictures are taken in an indoor studio and shown by a mannequin, the mannequin being the whole body, and so on. If the information can be effectively used to recommend commodity object information, the probability that the recommended information obtains the click rate of the user is relatively high.

Therefore, in specific implementation, feature extraction can be performed on the commodity object picture through some image processing algorithms. For example, the shooting scene including the picture, the presence or absence of the character model or the props, even the display range, the human body posture, the expression and the like of the character model can be trained in a deep learning mode and the like, and then the extraction of the corresponding features in the picture is realized through the trained model. In this way, the feature vector of the commodity object picture can be generated, and then in a specific recommendation algorithm, a specific recommendation scheme can be determined through the operation of the feature vector of the commodity object picture, the feature vector of the user and the like. The commodity object picture features are described through the feature vectors and are input into a recommendation algorithm for operation, so that the click rate of recommendation information can be improved to a certain extent.

However, when the image feature extraction is performed in the above manner, the number of features that can be obtained is generally limited, and accordingly, the dimension of the feature vector of the generated product object is generally on the order of tens or tens. Moreover, the specific features are usually some features of a comparative image, including the shooting scene, the presence or absence of the character model or the props, even the display range, the body posture, the expression and the like of the character model, and may further include the dominant hue, the brightness and the like of the picture. The character features are usually defined according to human comprehension, but in practice, a picture usually contains more features, which may be relatively abstract features, and may even be beyond the scope of human comprehension, discovery or definition. If the abstract features are added into the feature vector of the commodity object picture to generate a feature vector with higher dimensionality, the features of the picture can be expressed more completely and more abundantly. Furthermore, when the high-dimensional feature vector is input into a specific recommendation algorithm for operation, more precise and accurate recommendation information can be obtained, and a better click rate can be obtained for the recommendation information.

In order to achieve the above purpose, the embodiment of the present invention may perform the extraction of the commodity image feature through the neural network model, since the neural network generally includes multiple layers, each layer may include multiple neurons (where, the number of neurons in the input layer and the output layer is often fixed, and the middle layer may be freely specified, for example, for a three-layer neural network, there are 3 input neurons in the input layer, 2 neurons in the output layer, 4 neurons in the middle layer, or 5 neurons in the middle layer, and so on). The specific output result of the output layer is usually determined according to the training direction specified in the training process. For example, if a certain neural network is used to identify a scene of a certain picture, after a specific picture is input to the neural network, the output result of the output layer is whether the scene corresponding to the picture is street or greenhouse, or what the probabilities are. At the same time, the neural network also has some intermediate layers, and the activation value of each neuron in these intermediate layers may be various features extracted from the picture, or features obtained by weighted summation and nonlinear transformation of the neurons in the previous layer, which may be abstract features that human beings cannot understand or define, but are meaningful because these features can affect the output result of the output layer. Therefore, in the embodiment of the application, in the process of extracting the features of the commodity object picture by using the neural network, in addition to determining the feature values of the commodity object in one or more dimensions according to the output result of the output layer, the output values (generally referred to as activation values) of part or all of the intermediate layers may be obtained, and the activation values of the intermediate layers are also used as the feature values of the commodity object in more dimensions. In this way, the feature vector actually generated for the commodity object may include some dimension feature values that are relatively visualized and can be understood by human beings, and may also include some relatively abstract feature values, in other words, the abstract feature may be used as a supplement to the visual feature, so as to obtain a high-dimension feature vector for the commodity object. And the high-dimensional feature vector can be input into a specific recommendation algorithm to obtain a more refined and accurate recommendation result.

In specific implementation, from the perspective of system architecture, referring to fig. 1, the embodiment of the present application may mainly relate to a server and a client of a commodity object information service system. The server side can generate specific recommendation information and then send the recommendation information to the client side of the corresponding user, so that the user can browse the specific recommendation information through the client side. Specifically, the server may further provide a feature extraction module, where the module may store a plurality of neural network models, and each neural network model may be used to extract features included in the commodity object picture from different angles. For example, the neural network a is used to identify a commodity object subject region and a background region in a picture, the neural network B is used to identify a shooting scene category of the picture, and so on. Specifically, when feature information of a picture of a certain commodity object needs to be extracted, the same picture can be respectively input into a plurality of neural networks for feature extraction. In addition, the feature extraction module may further include a feature vector generation module, which may obtain an output result of the final output layer and an activation value of one or more intermediate layers from the plurality of neural networks, respectively, and may further generate a feature vector of a specific picture by using the output result and the activation value as feature values in a plurality of dimensions. The feature vector can be input into a specific recommendation algorithm module to obtain a specific recommendation result and provide the specific recommendation result to the client. Certainly, in practical applications, the high-dimensional feature vector about the commodity object generated by the scheme provided by the embodiment of the present application may also be applied to other scenes besides the recommendation system, for example, when commodity object information is delivered to a page, a higher user click rate can be obtained when a specific picture is used as a representative picture according to the feature vector information.

The following describes in detail specific implementations provided in embodiments of the present application.

Example one

First, the first embodiment provides an information processing method, and referring to fig. 2, the method may specifically include:

s201: acquiring at least one neural network model for carrying out feature extraction on the commodity object picture;

in the embodiment of the present application, a plurality of neural network models may be obtained, where such a model may be a model having a function of extracting features of a commodity object picture. Wherein the neural network model identifies that the object has a corresponding model class according to different characteristics. The reason why multiple models exist is that, for the same picture, the features to be extracted may be many-sided, and the features may not be completely mutually exclusive, so that it is difficult to extract the features through the same model. For this reason, feature extraction of the commodity object picture can be realized from various different angles through a plurality of different neural network models. For example, the model may specifically include a model for distinguishing an area where the subject of the commodity object is located from a background area, may further include a model for identifying whether the shooting scene belongs to a street photo or a booth photo, may further include a model for identifying a display range of a character model, and the like. In short, for the same picture to be subjected to feature extraction, the same picture can be respectively input into a plurality of different neural network models to perform feature extraction from different angles.

When implemented, the neural network model may be some existing models, for example, a model for recognizing a human face may exist in the prior art, and the like. Or, in more cases, the commodity object image may be obtained by training through a training sample, where the training sample may include information of a plurality of commodity object images and labeling information for labeling the commodity object images according to an influence factor of an image click rate. That is, the model is trained through the specific pictures and the labeled information, so that the trained model can be used for identifying the characteristics corresponding to the labeled information.

For example, the specific annotation information may include: and training the area corresponding to the commodity object main body in the sample picture and the area corresponding to the background. At this time, after inputting the picture of the specific commodity object into the trained model, the output result of the neural network model output layer may include: and a plurality of pixels in the target commodity object picture belong to the class information of the commodity object main body or the background. Specifically, assuming that the picture of a commodity object is a 100 × 100 pixel picture, the output of the model may be a 100 × 100 matrix, and the value of each element in the matrix may be 1 or 0, where 1 may indicate that the corresponding pixel belongs to the commodity object body, and 0 indicates that the corresponding pixel belongs to the background, and so on. In the specific implementation, information such as the dominant hue of the main body part of the commodity object can be factors influencing the click rate of the user, so after the output result is obtained, the pixel attributes of the pixels in the element categories can be determined according to the output result, the target pixel attributes are determined by comparing the pixel attributes of the pixels, and then the characteristic value of the data object picture in at least one dimension is determined according to the target pixel attributes. For example, in a specific implementation, according to the output result, color attribute information of a plurality of pixels belonging to a commodity object main body in the target commodity object picture may be determined, and dominant hue information of the commodity object main body portion may be determined, and then, dominant hue information of the commodity object main body portion may be determined as a feature value in one dimension in the feature vector.

Or, in another mode, the proportion of the main body part of the commodity object and the background part in the target commodity object picture and/or the quantity information of the connected regions may also be determined, so that the proportion and/or the quantity information of the connected regions may also be determined as a feature value in a part dimension in the feature vector, and the like.

In addition, the labeling information corresponding to the training sample may also be: training shooting scene category information corresponding to the sample picture; in this case, the output result of the neural network model output layer includes: and the category information of the shooting scene of the target commodity object picture. Specifically, the shooting scene category information may also have different category classification manners from different angles, for example, the shooting may include indoor shooting or outdoor shooting, shooting performed on a commodity object in a use state through a model person or a prop, or shooting performed on details of the commodity object without the model person or the prop. Wherein, under different classification modes, different neural network models can be used for feature extraction.

Moreover, for the training sample picture showing the using state of the commodity object through the model figure, the specific marking information may also include: characteristic information of the model figure. Specifically, the feature information of the model person may also have a plurality of different classification manners from different angles, for example, including: whether the whole/half body of the model character is included in the target commodity object picture, whether a face image of the model character is included, or posture characteristic information of the model character, or expression characteristic information of the model character, and the like. Similarly, different neural network models can be used for feature extraction in different partitioning modes.

S202: inputting a target commodity object picture into the neural network model to obtain an output result of an output layer of the neural network model and an activation value of at least one intermediate layer; the activation value of the middle layer represents feature information extracted from the commodity object picture by the middle layer of the neural network model in the process of obtaining an output result, or feature information obtained by performing weighted summation and nonlinear transformation on neurons in the upper layer;

after obtaining a plurality of neural network models, these models can be used to perform feature extraction on the picture of a specific target commodity object. The target commodity object may be a specific commodity object that needs to be recommended to the user, for example, a new item that a certain merchant puts on the shelf, and the like. Alternatively, the user may be provided with recommendation information such as "guess you like", or the user may be provided with a predetermined commodity object in the commodity pool. In summary, for each specific commercial object that may be recommended, pictures of the commercial object may be respectively input into a plurality of neural network models, and feature extraction is performed to generate feature vectors. The picture of the commodity object may be a picture used as a representative picture of the commodity object. The representative image is a picture representing a commodity object when the commodity object is placed on a certain page and displayed, and is displayed in a resource slot of the page.

In this embodiment, after a specific picture is input to the neural network model, the specifically obtained features may include not only an output result of an output layer of the neural network model, but also an activation value of an intermediate layer. That is, as shown in fig. 3, for a neural network model, it usually includes multiple layers, each layer including multiple neurons. Specifically, in a convolutional neural network, each layer may be referred to as a convolutional layer, and a convolutional layer generally includes several feature planes (featuremaps), each of which is composed of a number of neurons arranged in a rectangular shape. In the process of feature extraction through the neural network model, the neurons of each intermediate layer also respectively extract features of the picture, and the features extracted by the neurons may be some abstract features which cannot be understood by human beings. For this reason, in the embodiment of the present application, in addition to obtaining the output result of the output layer of the neural network model, the activation values of the intermediate layer may also be obtained, and these activation values may also be used as part of describing the picture features.

That is, in the embodiment of the present application, for a picture of a specific commodity object that needs to be recommended or the like, the picture may be input into a plurality of neural network models, and for each neural network model, an output result of a specific output layer and an activation value of at least one intermediate layer may be obtained.

And the output value of the specific output layer and the activation value of the intermediate layer are different according to different types of the specific neural network models. For example, in one case, the output results of the neural network model output layer may include: element category feature information of image pixels in the target commodity object picture, wherein the element category includes a commodity object main body or a background, and at this time, the activation value of the intermediate layer includes: in the process of obtaining the element category feature information, the intermediate layer of the neural network model extracts feature information from the target commodity object picture, or obtains feature information by performing weighted summation and nonlinear transformation on neurons in the upper layer. Or, the output result of the neural network model output layer comprises: the category characteristic information of the shooting scene of the target commodity object picture; at this time, the activation values of the intermediate layer include: in the process of obtaining the category characteristic information of the shooting scene, the intermediate layer of the neural network model extracts the characteristic information from the target commodity object picture or obtains the characteristic information obtained by weighting summation and nonlinear transformation on the neuron in the previous layer. Alternatively, the output result of the neural network model output layer may also include: for training sample pictures for displaying the using state of the commodity object through a model figure, the characteristic information of the model figure; at this time, the activation values of the intermediate layer include: in the process of obtaining the feature information of the model person, the intermediate layer of the neural network model extracts the feature information from the target commodity object picture or obtains the feature information by performing weighted summation and nonlinear transformation on neurons in the previous layer.

S203: and determining characteristic values on multiple dimensions according to the output result of the output layer and the activation value of the intermediate layer, and generating a characteristic vector of the commodity object picture.

After the output result of the specific neural network model and the activation value of the intermediate layer are obtained, the characteristic values in multiple dimensions can be respectively determined, and further, the characteristic vector of the commodity object picture is generated. Wherein, for different pictures, specific dimension information may be corresponding specifically when generating the feature vector. For example, the first dimension may represent brightness information of the picture, the second dimension represents dominant hue information of the main part of the commodity object in the picture, the third dimension represents shooting scene category of the picture, and so on; in the more backward dimension, there may be some abstract features, and the dimension identifications of these features may be replaced by numbers, etc., for example, dimension 500, dimension 700, etc. Of course, these dimensions denoted by numbers may specifically correspond to the activation values of the same neuron in the same middle layer of the same neural network. Through the dimension alignment, operations can be conveniently performed between feature vectors of different pictures, for example, the similarity of pictures is calculated.

In addition, in a specific implementation, the output result of the output layer and the activation value of the intermediate layer may be screened, for example, after the output value and the activation value of the intermediate layer are obtained from a plurality of models, the number of specifically obtained features may be very large, and at this time, the features may be first screened to filter some useless features, and then feature values in a plurality of dimensions are determined according to the screening result, so that the accuracy of the algorithm is improved.

It should be noted that, in the embodiment of the present application, the specific commodity object picture may include a still picture (a photo, etc.) associated with the commodity object, or may also include a picture extracted from a video associated with the commodity object, and the like. For the former, when recommending to a user specifically, the picture of the commodity object can be recommended to the client associated with the user, and for the latter, after generating the feature vector through the picture extracted from the video and matching with the user, the video of the commodity object can be recommended to the client associated with the user.

In a word, through the embodiment of the application, the characteristic extraction can be performed on the commodity object picture through the neural network model, the output result of the model output layer can be used as the characteristic value on a specific dimension in the picture characteristic vector, and the activation value of the model intermediate layer can be used as the characteristic value on more dimensions in the characteristic vector. In this way, a high-dimensional feature vector can be generated for the commodity object picture, wherein the feature vector can include not only the feature values in the dimensions that can be understood and defined by human beings, but also the feature values in the dimensions that can not be understood or accurately defined by human beings, so that the feature vector can express the features of the picture more completely. And furthermore, in the scenes of commodity object information recommendation and the like, more accurate recommendation results can be obtained, the click rate of recommendation information is improved, and the resource waste is reduced.

It should be noted that, in the embodiments of the present application, the user data may be used, and in practical applications, the user-specific personal data may be used in the scheme described herein within the scope permitted by the applicable law, under the condition of meeting the requirements of the applicable law and regulations in the country (for example, the user explicitly agrees, the user is informed, etc.).

Example two

The first embodiment provides a method for obtaining a specific commodity object picture feature vector, and in other application scenarios, a similar scheme may be used to obtain a user feature vector. The user feature vector is a vector for describing a feature of a user, and the user may be a consumer user, a buyer user, or the like in the commodity object information service system. In a scenario of recommending commodity object information to a user, or recommending other users having common preferences with the user to the user, etc., the user feature vector is generally required to be input into a specific matching algorithm for operation. The user feature vector may include feature values in multiple dimensions, the specific dimension may include basic feature values in the dimensions of the age, gender, occupation, and the like of the user, information such as commonalities that the user is interested in the commodity objects may be extracted according to information of the commodity objects related to the user historical behaviors (browsing, collecting, buying, purchasing, and the like), and the information in the dimensions may also be used as feature values in the specific dimension in the user feature vector to describe the user features.

When extracting specific commodity object feature information from the commodity objects associated with the user historical behavior information, the commodity object feature information may include some visual features of pictures associated with the commodity objects, such as the shooting scene category described above, whether the character models are included, and the like, in addition to basic information such as categories and brands of the commodity objects. When the visual features are acquired, the visual features can be acquired through a neural network model, and the output result of the output layer of the neural network model and the activation value of the middle layer can be acquired, so that the high-dimensional feature vector of the user is generated. The characteristic values in the dimension which can be understood and defined by human beings can be included, and the characteristic values in the dimension which cannot be understood or exactly defined by human beings can also be included, so that more comprehensive and more detailed description of the user characteristics is realized. So that the commodity object information recommended to the user or the information of other users is more accurate.

Specifically, referring to fig. 4, a second embodiment provides a method for determining user characteristic information, where the method specifically includes:

s401: acquiring picture information of a plurality of commodity objects associated with historical behavior information of a user;

the picture of the commodity object may specifically refer to a representative picture of the commodity object, that is, a picture serving as a head picture of the commodity object in a detail page of the commodity object, or a picture displayed in the resource slot when the commodity object is placed on a specific commodity object list page. Since if a user clicks a certain commodity object among a plurality of commodity objects, the commodity object attracts the user due to factors such as the category, the brand, and the like, the representative picture of the commodity object may arouse the user's interest. Therefore, the characteristics of the commodity object picture can be described as part of the user characteristics.

S402: acquiring at least one neural network model for carrying out feature extraction on the commodity object picture;

s403: respectively inputting pictures of a plurality of commodity objects into the neural network model to obtain an output result of an output layer of the neural network model and an activation value of at least one intermediate layer; the activation value of the middle layer represents feature information extracted from the commodity object picture by the middle layer of the neural network model in the process of obtaining an output result, or feature information obtained by performing weighted summation and nonlinear transformation on neurons in the upper layer;

s404: and determining characteristic values on multiple dimensions by counting the output results of the output layers respectively corresponding to the pictures of the commodity objects and the activation values of the middle layer so as to generate the characteristic vector of the user.

After the output results of the output layers corresponding to the pictures of the plurality of commodity objects and the activation values of the intermediate layer are obtained, statistics may be performed, for example, statistics may be performed on the number or frequency of occurrence of various features, the degree of interest of the user on the features in the corresponding dimension is determined, and then information such as the degree of interest corresponding to the dimensions is used as feature values to generate specific user feature vectors.

EXAMPLE III

The first embodiment provides a method for obtaining a feature vector of a picture of a specific commodity object, and after the feature vector of the picture of the specific commodity object is determined, the method can be applied to scenes such as recommendation of the specific commodity object. Specifically, the second embodiment provides a specific application scenario, which may be a scenario of recommendation information such as "guess you like" provided through a client top page and the like. That is, when the user accesses the client top page, a "guess you like" block may be provided in the top page, in the block, a plurality of resource slots may be provided, and each resource slot may be associated with the information of the specific recommended goods object. The recommended commodity object information is usually determined by matching the characteristics of the current visitor user with the commodity objects in the commodity object library. The characteristics of the commodity object can be described with the characteristic vector of the commodity object, wherein the characteristics can include the characteristics corresponding to the specific commodity object picture, the characteristics of the commodity object picture can include some visualized characteristics, and in addition, the characteristics can also include abstract characteristics obtained through the activation value of the neural network intermediate layer, so that the high-dimensional characteristic vector of the commodity object is generated. And then inputting the characteristic vector of the commodity object and the user vector into a recommendation algorithm for matching operation, and determining the commodity object which is specifically suitable for being recommended to the current user. For the user vector, a vector expression manner in the prior art may be adopted, or a high-dimensional feature vector in the foregoing second embodiment may also be adopted for expression, and so on.

Specifically, referring to fig. 5, a third embodiment provides a method for recommending a commodity object, which may specifically include:

s501: determining a target recommendation user of the commodity object and a source database of the commodity object recommendation information;

the specific target recommendation user may be a user who wants to obtain the recommendation information of the commodity object, and various situations may be specific. For example, the access request may be a user initiating an access request to a certain target page, and if a layout block such as "guess you like" exists in the target page, the access request initiated to the target page may also be simultaneously used as a request initiated by the user to acquire recommendation information. The specific recommendation information source database may also depend on the specific application scenario, for example, it may be the full database of the system, or some small database of some businesses, merchants, etc.

S502: acquiring feature vector information of the user and feature vector information of a plurality of commodity objects in the source database, wherein the feature vectors of the commodity objects comprise feature values of pictures of the commodity objects in a plurality of dimensions, and the feature values comprise: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the intermediate layer is the characteristic information extracted from the picture of the commodity object by the intermediate layer of the neural network model in the process of obtaining the output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on the neuron on the upper layer;

s503: and inputting the feature vector information of the user and the feature vector information of the plurality of commodity objects into a recommendation algorithm to predict click rate information obtained after recommending the commodity objects to the user.

The specific recommendation algorithm is not the focus of attention in the embodiments of the present application, and therefore, will not be described in detail herein.

After obtaining the specific click rate prediction result, at least one commodity object with a click rate meeting the condition can be determined from the plurality of commodity objects so as to be recommended to the user.

Example four

The fourth embodiment provides another specific recommendation scenario. Specifically, given a specific commodity object, one or more users most likely to click on the commodity object can be predicted from a plurality of users, and the commodity object can be recommended to the users. The situation can mainly occur in scenes such as new product recommendation of merchants, for example, a merchant puts a new item of commodity object on shelf, and some users need to be selected for directional recommendation. In this case, the feature vector of the commodity object may be matched with the user feature vectors of the plurality of users, so as to determine probability information clicked by the user when the commodity object is recommended to each user. And then determines which users are specifically recommended to. Specifically, referring to fig. 6, a fourth embodiment provides a method for recommending a commodity object, where the method may specifically include:

s601: determining picture information of a commodity object to be recommended;

s602: acquiring feature vector information of the commodity object, wherein the feature vector of the commodity object comprises feature values of the picture in multiple dimensions, and the feature values comprise: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

s603: acquiring feature vector information of a plurality of users;

s604: and inputting the characteristic vector information of the commodity object and the characteristic vector information of the users into a recommendation algorithm to predict click rate information obtained after the commodity object to be recommended is recommended to the users.

In the concrete implementation, the commodity object information to be recommended can be recommended to the users with the click rate predicted values meeting the conditions according to the predicted click rate information.

EXAMPLE five

In addition to the fact that the recommendation of a specific commodity object to a user can be achieved by matching the commodity object feature vector with the feature vector of the user, the commodity object feature vector generated in the embodiment of the present application can also be applied to scenes such as similarity calculation between the commodity object and the commodity object. For example, in one scenario, information of multiple commodity objects may be shown in multiple resource positions in some commodity object list pages, where each resource position may be operated by an operation option such as "find similar", and a user may obtain information of more other commodity objects similar to the commodity object in the resource position by clicking the operation option. At this time, similarity calculation is performed between the feature vector of the commodity object in the resource position and the feature vectors of other commodity objects to determine the similarity between the commodity objects. Or, when the current user performs operations such as browsing a detailed page of a certain commodity object, adding to a to-be-purchased set, or collecting, the system may also provide a "see-and-see" function, provide more commodity object information similar to the commodity object operated by the user for the user to select, and the like. In this case, the similarity comparison between the product objects is also involved, and the determination can be performed by the calculation of the feature vectors of the product objects. The feature vector of the commodity object can also be implemented in a high-dimensional vector manner by using the scheme provided in the embodiment of the present application, which may include an abstract feature obtained by an intermediate layer activation value of a neural network model. Specifically, the fifth embodiment provides a method for providing a commodity object, and referring to fig. 7, the method may specifically include:

s701: in the process of providing similar commodity object information according to a target commodity object, obtaining feature vector information of the target commodity object and feature vector information of a plurality of commodity objects in a source database; the feature vector of the commodity object comprises feature values of pictures of the commodity object in multiple dimensions, wherein the feature values comprise: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

specifically, in the process of displaying a target page including at least one commodity object information, a request for obtaining similar commodity object information for a specified commodity object is received, and the specified commodity object is determined as the target commodity object. Or receiving an operation request for performing detail page browsing on the specified commodity object, or adding the specified commodity object to a to-be-purchased set or collecting the specified commodity object, and determining the specified commodity object as the target commodity object.

S702: and comparing the similarity of the characteristic vector information of the target commodity object with the characteristic vector information of a plurality of commodity objects in a source database to provide commodity object information with the similarity meeting the condition of the specified commodity object.

EXAMPLE six

In addition to being used in scenes such as recommendation of commodity objects or provision of information of similar commodity objects, the high-dimensional feature vector information of specific commodity object pictures can be applied to scenes for optimizing representative pictures of commodity objects. Specifically, since the representative picture of the commodity object can be usually displayed in the resource slot of a specific page, and the characteristics of the picture may affect the click of the user. For example, for a user, if one of the pictures of the commodity object is shown in the resource slot, the commodity object may not be clicked, but if the picture is changed to another picture, the commodity object may be clicked, and so on. For such a situation, in the embodiment of the present application, when a specific commodity object is dropped into a certain target page, multiple selectable pictures may be associated with the commodity object, where which picture is specifically displayed as a representative picture in a specific resource slot may be determined according to the characteristics of a specific visitor user. That is to say, after receiving an access request of a specific visitor user to a target page, matching operation can be performed on the feature vector of the user and the feature vectors of the pictures of the commodity object, and when which picture is displayed is calculated, a higher click probability can be obtained for the commodity object, and then the picture can be displayed in a specific resource position as a representative picture of the commodity object. Therefore, the 'thousands of people and thousands of faces' on the commodity object representation diagram level can be realized, and the click rate of the commodity object is improved. The feature vector of the specific picture may be a high-dimensional feature vector containing some abstract features generated in the embodiment of the present application, and the feature vector of the user may be low-dimensional or high-dimensional, so that more accurate matching may be achieved.

Specifically, the sixth embodiment provides a method for providing information of a commodity object, and referring to fig. 8, the method may specifically include:

s801: receiving an access request to a target page, wherein the target page is associated with a plurality of resource positions and is used for displaying information of a plurality of commodity objects, and the commodity objects are associated with a plurality of different pictures;

the target page may be various, for example, a client top page, or a home page of some event venues, a channel home page, and the like.

S802: acquiring feature vector information corresponding to the multiple different pictures and feature vector information of the visitor user; wherein, the feature vector corresponding to the picture comprises feature values of the picture in multiple dimensions, and the feature values comprise: after the pictures are input into at least one neural network model for carrying out feature extraction on the commodity object pictures, outputting the output result of the layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

s803: respectively inputting the feature vector information of the visitor user and the feature vector information corresponding to the multiple pictures of the commodity object into a prediction algorithm so as to predict click rate information respectively obtained after the multiple pictures of the commodity object are displayed to the user;

s804: and according to the prediction result, displaying the picture with the click rate meeting the condition as a representative picture of the corresponding commodity object in the resource position corresponding to the target page.

For the parts that are not described in detail in the second to sixth embodiments, reference may be made to the description in the first embodiment, and details are not repeated here.

EXAMPLE seven

As described above, the neural network model or other algorithm models may be used to analyze the picture of the commodity object to determine the display scene category information associated with the picture of the commodity object, for example, the display scene category information may include a show scene, a coffee drink scene, an indoor studio scene, an outdoor street scene, a sports scene, and the like. For the user, when specific user feature information is acquired, display scene information in which the user is interested may also be acquired. For example, by analyzing pictures of commodity objects historically browsed by the user, and analyzing the display scene information corresponding to the pictures, the display scene information interested by the user can be determined, and the like. Therefore, when information recommendation is specifically performed on the user, recommendation can be performed by combining the display scene type information associated with the specific commodity object picture. Specifically, in the seventh embodiment, an information recommendation method is provided, referring to fig. 9, where the method specifically includes:

s901: determining a picture associated with the target commodity object;

s902: determining display scene category information corresponding to the commodity object in the picture by performing feature analysis on the picture;

when the feature analysis is performed on the picture, a neural network model trained in advance or a common algorithm model can be used for performing the feature analysis, and details are not described here.

S903: determining a user set, and acquiring user characteristic information corresponding to a plurality of users in the user set respectively, wherein the user characteristic information comprises display scene category information which is interesting to the users;

regarding the obtaining of the user features, the first embodiment is also correspondingly described, and the description thereof is omitted here.

S904: determining a target user according to the display scene category information corresponding to the picture and the display scene category information interested by the user;

s905: and recommending the picture to a client associated with the target user.

Example eight

This embodiment eight is similar to the embodiment seven, and the specifically recommended object is no longer a picture of the commodity object but a video of the commodity object. Such video may be of various origins, and may include, for example, video captured and uploaded by merchant users, or may be video captured in a live scene, and so forth. In this case, after the video to be recommended is determined, a plurality of frames of images can be extracted from the video, and the display scene category information corresponding to the video can be determined by analyzing the image features. The video may then be recommended to users interested in the scene category. Specifically, referring to fig. 10, an eighth embodiment specifically provides an information recommendation method, which specifically includes:

s1001: determining a video associated with the target commodity object;

s1002: extracting at least one frame of image from the video, and determining display scene category information corresponding to the commodity object in the video by performing feature analysis on the image;

s1003: determining a user set, and acquiring user characteristic information corresponding to a plurality of users in the user set respectively, wherein the user characteristic information comprises display scene category information which is interesting to the users;

s1004: determining a target user according to the display scene category information corresponding to the video and the display scene category information which is interested by the user;

s1005: and recommending the video to the client associated with the target user.

Corresponding to the first embodiment, an embodiment of the present application further provides an information processing apparatus, and referring to fig. 11, the apparatus may specifically include:

a model obtaining unit 1101, configured to obtain at least one neural network model for performing feature extraction on a commodity object picture;

an analysis unit 1102, configured to input a target commodity object picture into the neural network model, and obtain an output result of an output layer of the neural network model and an activation value of at least one intermediate layer; the activation value of the middle layer represents feature information extracted from the commodity object picture by the middle layer of the neural network model in the process of obtaining an output result, or feature information obtained by performing weighted summation and nonlinear transformation on neurons in the upper layer;

a picture feature vector generating unit 1103, configured to determine feature values in multiple dimensions according to the output result of the output layer and the activation value of the intermediate layer, and generate a feature vector of the commodity object picture.

The neural network model is obtained by training a training sample, wherein the training sample comprises information of a plurality of commodity object pictures and marking information for marking the commodity object pictures according to the picture click rate; the labeling information includes: and the marking information is used for distinguishing the commodity object area and the background area in the training sample picture, or shooting scene type information corresponding to the training sample picture, or characteristic information of a model figure included in the training sample picture.

Specifically, the output result of the neural network model output layer includes: element category feature information of image pixels in the target commodity object picture, wherein the element category comprises a commodity object main body or a background; at this time, the activation values of the intermediate layer include: in the process of obtaining the element category feature information, the intermediate layer of the neural network model extracts feature information from the target commodity object picture, or obtains feature information by performing weighted summation and nonlinear transformation on neurons in the upper layer.

In a specific implementation, the apparatus may further include:

a pixel attribute determining subunit, configured to determine a pixel attribute of each pixel in the element category according to the output result;

a target pixel attribute determining subunit, configured to determine a target pixel attribute by comparing pixel attributes of the pixels;

in this case, the picture feature vector generation unit may be specifically configured to: and determining the characteristic value of the data object picture on at least one dimension according to the target pixel attribute.

Specifically, the pixel attribute determining subunit may be specifically configured to: determining color attribute information of a plurality of pixels corresponding to the commodity object main body in the target commodity object picture according to the output result;

the target pixel attribute determining subunit may be specifically configured to: determining dominant hue information of the merchandise object body portion;

the picture feature vector generation unit may be specifically configured to: and determining the main tone information of the main body part of the commodity object as a characteristic value in one dimension in the characteristic vector.

Or, in another mode, the target pixel attribute determining subunit may be specifically configured to: determining the proportion of the main body part of the commodity object and the background part of the commodity object in the target commodity object picture and/or the quantity information of the connected areas;

the picture feature vector generation unit may be specifically configured to: and determining the occupation ratio and/or the quantity information of the connected regions as characteristic values on partial dimensions in the characteristic vector.

In addition, the output result of the neural network model output layer comprises: the category characteristic information of the shooting scene of the target commodity object picture; at this time, the activation values of the intermediate layer include: in the process of obtaining the category characteristic information of the shooting scene, the intermediate layer of the neural network model extracts the characteristic information from the target commodity object picture or obtains the characteristic information obtained by weighting summation and nonlinear transformation on the neuron in the previous layer. Wherein the shooting scene category information: indoor shooting or outdoor shooting, shooting of commodity object using states through model characters or props, or shooting of commodity object details without the model characters or props.

Or, the output result of the neural network model output layer comprises: for training sample pictures for displaying the using state of the commodity object through a model figure, the characteristic information of the model figure; the activation values of the intermediate layer include: in the process of obtaining the feature information of the model person, the intermediate layer of the neural network model extracts the feature information from the target commodity object picture or obtains the feature information by performing weighted summation and nonlinear transformation on neurons in the previous layer. Wherein the characteristic information of the model character comprises: whether the whole body/half body of the model character is included in the target commodity object picture, whether a face image of the model character is included in the target commodity object picture, or whether the posture characteristic information of the model character is included in the target commodity object picture, or whether the facial image of the model character is included in the target commodity object picture, or whether the posture characteristic information of the model character is included in the target commodity object picture, or whether the expression characteristic information of the model.

In a specific implementation, the apparatus may further include:

the characteristic screening unit is used for screening the output result of the output layer and the activation value of the middle layer;

the picture feature vector generation unit may be specifically configured to: and determining characteristic values on multiple dimensions according to the screening result.

Corresponding to the second embodiment, an embodiment of the present application further provides an apparatus for determining user characteristic information, and referring to fig. 12, the apparatus may include:

a picture information determining unit 1201, configured to determine picture information of a plurality of commodity objects associated with historical behavior information of a user;

a model obtaining unit 1202, configured to obtain at least one neural network model for performing feature extraction on a commodity object picture;

an analyzing unit 1203, configured to input pictures of a plurality of commodity objects into the neural network model, respectively, to obtain an output result of an output layer of the neural network model and an activation value of at least one intermediate layer; the activation value of the middle layer represents characteristic information extracted from the commodity object picture by the middle layer of the neural network model in the process of obtaining an output result;

a user feature vector generating unit 1204, configured to determine feature values in multiple dimensions by performing statistics on output results of output layers corresponding to the pictures of the multiple commodity objects respectively and the activation value of the intermediate layer, so as to generate a feature vector of the user.

Corresponding to the three phases of the embodiment, the embodiment of the present application further provides a device for recommending commodity object information, referring to fig. 13, the device may include:

an information determining unit 1301, configured to determine a target recommended user of a commodity object and a source database of commodity object recommendation information;

a feature vector obtaining unit 1302, configured to obtain feature vector information of the user and feature vector information of a plurality of commodity objects in the source database, where the feature vectors of the commodity objects include feature values of pictures of the commodity objects in multiple dimensions, and the feature values include: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the intermediate layer is the characteristic information extracted from the picture of the commodity object by the intermediate layer of the neural network model in the process of obtaining the output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on the neuron on the upper layer;

the predicting unit 1303 is configured to input the feature vector information of the user and the feature vector information of the plurality of commodity objects into a recommendation algorithm to predict click rate information obtained after recommending the commodity objects to the user.

In a specific implementation, the apparatus may further include:

and the recommending unit is used for determining at least one commodity object with the click rate meeting the condition from the plurality of commodity objects according to the predicted click rate information so as to recommend the commodity object to the user.

Corresponding to the fourth embodiment, an embodiment of the present application further provides a commodity object recommending apparatus, referring to fig. 14, the apparatus may include:

a picture information determining unit 1401 for determining picture information of a commodity object to be recommended;

a commodity feature vector obtaining unit 1402, configured to obtain feature vector information of the commodity object, where the feature vector of the commodity object includes feature values of the picture in multiple dimensions, and the feature values include: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

a user feature vector acquisition unit 1403 having feature vector information on acquiring a plurality of users;

the predicting unit 1404 is configured to input the feature vector information of the commodity object and the feature vector information of the multiple users into a recommendation algorithm, so as to predict click rate information obtained after recommending the commodity object to be recommended to the users.

When the concrete implementation is carried out, the method can further comprise the following steps:

and the prediction unit is used for recommending the commodity object information to be recommended to the users with click rate predicted values meeting the conditions according to the predicted click rate information.

Corresponding to the fifth embodiment, the embodiment of the present application further provides an apparatus for providing a commodity object, referring to fig. 15, the apparatus may include:

a feature vector acquisition unit 1501, configured to acquire feature vector information of a target commodity object and feature vector information of a plurality of commodity objects in a source database in a process of providing similar commodity object information according to the target commodity object; the feature vector of the commodity object comprises feature values of pictures of the commodity object in multiple dimensions, wherein the feature values comprise: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

a similar commodity object information providing unit 1502 is configured to provide commodity object information whose similarity with the specified commodity object meets the condition by performing similarity comparison between the feature vector information of the target commodity object and the feature vector information of the plurality of commodity objects in the source database.

In addition, the apparatus may further include:

the first request receiving unit is used for receiving a request for obtaining similar commodity object information aiming at a specified commodity object in the process of displaying a target page comprising at least one commodity object information, and determining the specified commodity object as the target commodity object.

Or, the second request receiving unit is configured to receive an operation request for performing detail page browsing, or adding to a to-be-purchased collection, or collecting on a specified commodity object, and determine the specified commodity object as the target commodity object.

Corresponding to the sixth embodiment, the present application further provides an apparatus for providing information of a commodity object, and referring to fig. 16, the apparatus may include:

an access request receiving unit 1601, configured to receive an access request for a target page, where the target page is associated with a plurality of resource slots and is used to display information of a plurality of commodity objects, where the commodity objects are associated with a plurality of different pictures;

a feature vector obtaining unit 1602, configured to obtain feature vector information corresponding to each of the multiple different pictures and feature vector information of the visitor user; wherein, the feature vector corresponding to the picture comprises feature values of the picture in multiple dimensions, and the feature values comprise: after the pictures are input into at least one neural network model for carrying out feature extraction on the commodity object pictures, outputting the output result of the layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

a predicting unit 1603, configured to input the feature vector information of the visitor user and the feature vector information corresponding to the multiple pictures of the commodity object into a prediction algorithm, respectively, so as to predict click rate information obtained after the multiple pictures of the commodity object are displayed to the user, respectively;

and a representative picture determining unit 1604, configured to, according to the prediction result, display the picture with the click rate meeting the condition as a representative picture of the corresponding commodity object in the resource slot corresponding to the target page.

Corresponding to the seventh embodiment, an embodiment of the present application further provides an information recommendation apparatus, and referring to fig. 17, the apparatus may include:

a picture determination unit 1701 for determining a picture associated with the target commodity object;

a display scene determining unit 1702, configured to determine, by performing feature analysis on the picture, display scene category information corresponding to the commodity object in the picture;

a user characteristic information obtaining unit 1703, configured to determine a user set, and obtain user characteristic information corresponding to each of a plurality of users in the user set, where the user characteristic information includes display scene category information in which the user is interested;

a target user determining unit 1704, configured to determine a target user according to the display scene category information corresponding to the picture and the display scene category information in which the user is interested;

a picture recommending unit 1705, configured to recommend the picture to the client associated with the target user.

Corresponding to the eighth embodiment, an embodiment of the present application further provides an information recommendation apparatus, and referring to fig. 18, the apparatus may include:

a video determining unit 1801, configured to determine a video associated with the target commodity object;

a display scene determining unit 1802, configured to extract at least one frame of image from the video, and determine display scene category information corresponding to a commodity object in the video by performing feature analysis on the image;

a user characteristic information obtaining unit 1803, configured to determine a user set, and obtain user characteristic information corresponding to each of a plurality of users in the user set, where the user characteristic information includes display scene category information in which the user is interested;

a target user determining unit 1804, configured to determine a target user according to the display scene category information corresponding to the video and the display scene category information in which the user is interested;

a video recommending unit 1805, configured to recommend the video to the client associated with the target user.

In addition, an embodiment of the present application further provides an electronic device, including:

one or more processors; and

a memory associated with the one or more processors for storing program instructions that, when read and executed by the one or more processors, perform operations comprising:

Alternatively, the first and second electrodes may be,

determining picture information of a commodity object to be recommended;

obtaining feature vector information of the commodity object, wherein the feature vector of the commodity object comprises feature values of the picture in multiple dimensions, and the feature values comprise: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

acquiring feature vector information of a plurality of users;

Alternatively, the first and second electrodes may be,

in the process of providing similar commodity object information according to a target commodity object, obtaining feature vector information of the target commodity object and feature vector information of a plurality of commodity objects in a source database; the feature vector of the commodity object comprises feature values of pictures of the commodity object in multiple dimensions, wherein the feature values comprise: after the pictures of the commodity objects are input into at least one neural network model for carrying out feature extraction on the pictures of the commodity objects, outputting the output result of a layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

Alternatively, the first and second electrodes may be,

obtaining feature vector information corresponding to the multiple different pictures and feature vector information of the visitor user; wherein, the feature vector corresponding to the picture comprises feature values of the picture in multiple dimensions, and the feature values comprise: after the pictures are input into at least one neural network model for carrying out feature extraction on the commodity object pictures, outputting the output result of the layer and the activation value of at least one intermediate layer; the activation value of the middle layer is the characteristic information extracted from the picture of the commodity object by the neural network model in the process of obtaining an output result, or the characteristic information obtained by carrying out weighted summation and nonlinear transformation on neurons in the upper layer;

Alternatively, the first and second electrodes may be,

determining a picture associated with the target commodity object;

and recommending the picture to a client associated with the target user.

Alternatively, the first and second electrodes may be,

determining a video associated with the target commodity object;

and recommending the video to the client associated with the target user.

Fig. 19 illustrates an architecture of electronic devices that may include, in particular, a processor 1910, a video display adapter 1911, a disk drive 1912, an input/output interface 1913, a network interface 1914, and a memory 1920. The processor 1910, the video display adapter 1911, the disk drive 1912, the input/output interface 1913, the network interface 1914, and the memory 1920 are communicatively coupled via a communication bus 1930.

The processor 1910 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an Application Specific Integrated Circuit (ASIC), or one or more Integrated circuits, and is configured to execute related programs to implement the technical solution provided in the present Application.

The Memory 1920 may be implemented in the form of a ROM (Read Only Memory), a RAM (Random Access Memory), a static storage device, a dynamic storage device, or the like. The memory 1920 may store an operating system 1921 for controlling operations of the electronic device 1900, and a Basic Input Output System (BIOS) for controlling low-level operations of the electronic device 1900. In addition, a web browser 1923, a data storage management system 1924, an information processing system 1925, and the like may also be stored. Information handling system 1925 may be an application that implements the operations of the steps described above in this embodiment of the present application. In summary, when the technical solution provided in the present application is implemented by software or firmware, the relevant program code is stored in the memory 1920 and called by the processor 1910 for execution.

The input/output interface 1913 is used for connecting an input/output module to input and output information. The i/o module may be configured as a component in a device (not shown) or may be external to the device to provide a corresponding function. The input devices may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices may include a display, a speaker, a vibrator, an indicator light, etc.

The network interface 1914 is used to connect a communication module (not shown in the drawings) to implement communication interaction between the present apparatus and other apparatuses. The communication module can realize communication in a wired mode (such as USB, network cable and the like) and also can realize communication in a wireless mode (such as mobile network, WIFI, Bluetooth and the like).

Bus 1930 includes a path that allows information to be transferred between various components of the device, such as processor 1910, video display adapter 1911, disk drive 1912, input/output interface 1913, network interface 1914, and memory 1920.

It should be noted that although the above devices only show the processor 1910, the video display adapter 1911, the disk drive 1912, the input/output interface 1913, the network interface 1914, the memory 1920, the bus 1930 and the like, in a specific implementation, the devices may also include other components necessary for normal operation. Furthermore, it will be understood by those skilled in the art that the apparatus described above may also include only the components necessary to implement the solution of the present application, and not necessarily all of the components shown in the figures.

From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present application.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the system or system embodiments are substantially similar to the method embodiments and therefore are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for related points. The above-described system and system embodiments are only illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

The information processing method, the information processing apparatus, and the electronic device provided by the present application are introduced in detail, and a specific example is applied in the present application to explain the principle and the implementation of the present application, and the description of the above embodiment is only used to help understand the method and the core idea of the present application; meanwhile, for a person skilled in the art, according to the idea of the present application, the specific embodiments and the application range may be changed. In view of the above, the description should not be taken as limiting the application.

Claims

1. An information processing method characterized by comprising:

2. The method of claim 1,

the neural network model is obtained by training a training sample, wherein the training sample comprises a plurality of commodity object picture information and marking information for marking the commodity object picture according to the picture click rate;

the labeling information includes: and the marking information is used for distinguishing the commodity object area and the background area in the training sample picture, or shooting scene type information corresponding to the training sample picture, or characteristic information of a model figure included in the training sample picture.

3. The method of claim 1,

the output result of the neural network model output layer comprises: element category feature information of image pixels in the target commodity object picture, wherein the element category comprises a commodity object main body or a background;

the activation values of the intermediate layer include: in the process of obtaining the element category feature information, the intermediate layer of the neural network model extracts feature information from the target commodity object picture, or obtains feature information by performing weighted summation and nonlinear transformation on neurons in the upper layer.

4. The method of claim 3, further comprising:

determining the pixel attribute of each pixel in the element category according to the output result;

determining a target pixel attribute by comparing pixel attributes of the pixels; the determining the feature values in the multiple dimensions according to the output result of the output layer and the activation value of the intermediate layer includes:

and determining the characteristic value of the data object picture on at least one dimension according to the target pixel attribute.

5. The method of claim 4,

the determining the pixel attribute of each pixel in the element category according to the output result includes:

determining color attribute information of a plurality of pixels corresponding to the commodity object main body in the target commodity object picture according to the output result;

the determining the target pixel attribute by comparing the pixel attributes of the pixels includes:

determining dominant hue information of the merchandise object body portion;

the determining a feature value of the data object picture in at least one dimension according to the target pixel attribute comprises:

and determining the main tone information of the main body part of the commodity object as a characteristic value in one dimension in the characteristic vector.

6. The method of claim 3,

determining the proportion of the main body part of the commodity object and the background part of the commodity object in the target commodity object picture and/or the quantity information of the connected areas;

and determining the occupation ratio and/or the quantity information of the connected regions as characteristic values on partial dimensions in the characteristic vector.

7. The method of claim 1,

the output result of the neural network model output layer comprises: the category characteristic information of the shooting scene of the target commodity object picture;

the activation values of the intermediate layer include: in the process of obtaining the category characteristic information of the shooting scene, the intermediate layer of the neural network model extracts the characteristic information from the target commodity object picture or obtains the characteristic information obtained by weighting summation and nonlinear transformation on the neuron in the previous layer.

8. The method of claim 7,

the shooting scene category information: indoor shooting or outdoor shooting, shooting of commodity object using states through model characters or props, or shooting of commodity object details without the model characters or props.

9. The method of claim 1,

the output result of the neural network model output layer comprises: for training sample pictures for displaying the using state of the commodity object through a model figure, the characteristic information of the model figure;

the activation values of the intermediate layer include: in the process of obtaining the feature information of the model person, the intermediate layer of the neural network model extracts the feature information from the target commodity object picture or obtains the feature information by performing weighted summation and nonlinear transformation on neurons in the previous layer.

10. The method of claim 9,

the characteristic information of the model character comprises: whether the whole body/half body of the model character is included in the target commodity object picture, whether a face image of the model character is included in the target commodity object picture, or whether the posture characteristic information of the model character is included in the target commodity object picture, or whether the facial image of the model character is included in the target commodity object picture, or whether the posture characteristic information of the model character is included in the target commodity object picture, or whether the expression characteristic information of the model.

11. The method of any one of claims 1 to 10, further comprising:

screening the output result of the output layer and the activation value of the intermediate layer;

the determining feature values in a plurality of dimensions includes:

and determining characteristic values on multiple dimensions according to the screening result.

12. A method for determining user characteristic information, comprising:

13. A method for recommending a commodity object, comprising:

14. The method of claim 13, further comprising:

and determining at least one commodity object with a click rate meeting the condition from the plurality of commodity objects according to the predicted click rate information so as to recommend the commodity object to the user.

15. A method for recommending a commodity object, comprising:

determining picture information of a commodity object to be recommended;

acquiring feature vector information of a plurality of users;

16. The method of claim 15, further comprising:

and recommending the commodity object information to be recommended to users with click rate predicted values meeting conditions according to the predicted click rate information.

17. A method for providing a commodity object, comprising:

18. The method of claim 17, further comprising, prior to the method:

in the process of displaying a target page including at least one commodity object information, receiving a request for obtaining similar commodity object information for a specified commodity object, and determining the specified commodity object as the target commodity object.

19. The method of claim 17, further comprising, prior to the method:

receiving an operation request for browsing a detail page or adding to a to-be-purchased set or collecting a specified commodity object, and determining the specified commodity object as the target commodity object.

20. A method of providing merchandise object information, comprising:

21. An information recommendation method, comprising:

determining a picture associated with the target commodity object;

and recommending the picture to a client associated with the target user.

22. An information recommendation method, comprising:

determining a video associated with the target commodity object;

and recommending the video to the client associated with the target user.

23. An information processing apparatus characterized by comprising:

24. An apparatus for determining user characteristic information, comprising:

25. A commodity object information recommendation device characterized by comprising:

26. A commodity object recommending apparatus, comprising:

27. An apparatus for providing a commodity object, comprising:

28. An apparatus for providing commodity object information, comprising:

29. An information recommendation apparatus, comprising:

30. An information recommendation apparatus, comprising:

31. An electronic device, comprising:

one or more processors; and

32. An electronic device, comprising:

one or more processors; and

33. An electronic device, comprising:

one or more processors; and

34. An electronic device, comprising:

one or more processors; and

determining picture information of a commodity object to be recommended;

acquiring feature vector information of a plurality of users;

35. An electronic device, comprising:

one or more processors; and

36. An electronic device, comprising:

one or more processors; and

37. An electronic device, comprising:

one or more processors; and

determining a picture associated with the target commodity object;

and recommending the picture to a client associated with the target user.

38. An electronic device, comprising:

one or more processors; and

determining a video associated with the target commodity object;

and recommending the video to the client associated with the target user.