CN110209916B - Method and device for recommending point of interest images - Google Patents
Method and device for recommending point of interest images Download PDFInfo
- Publication number
- CN110209916B CN110209916B CN201810111415.4A CN201810111415A CN110209916B CN 110209916 B CN110209916 B CN 110209916B CN 201810111415 A CN201810111415 A CN 201810111415A CN 110209916 B CN110209916 B CN 110209916B
- Authority
- CN
- China
- Prior art keywords
- image
- candidate
- category
- recommended
- interest
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9537—Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Abstract
The application discloses a method and a device for recommending images of interest points, wherein for candidate images in a candidate image set of the interest points, image categories of the candidate images are obtained by utilizing a classification model respectively, image ranking values of the candidate images are obtained by utilizing a ranking model, further, the categories of the interest points are obtained, the candidate images corresponding to the image categories with the highest category relevance degree of the interest points are obtained in the candidate image set of the interest points and serve as images to be recommended, and the recommended images of the interest points are determined according to the image ranking values of the images to be recommended. The method and the device comprehensively consider the relevance between the category of the candidate image and the category of the interest point and the sorting score value of the candidate image, and realize recommendation of the image with high relevance for the interest point.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for recommending a point of interest image.
Background
A POI (Point of Interest) is a term in the field of geographic information, and generally refers to any geographic object that can be abstracted as a Point, which may be a hospital, school, store, even a transit stop board, etc. Each POI may correspond to a set of candidate images, the set comprising a large number of candidate images.
In some scenarios, when presenting a POI to a user, it is desirable to present an image relating to the POI. As shown in fig. 1a and 1b, assuming that a POI is a restaurant, fig. 1a shows a dish image of the restaurant, fig. 1b is a picture of a scene taken inside the restaurant, and fig. 1a and 1b are higher in association with the POI and should be recommended preferentially than those of fig. 1a and 1 b. In order to show an image with high association with a POI to a user, a technical scheme for recommending an image with high association to the POI is urgently needed.
Disclosure of Invention
In view of this, the present application provides a method and an apparatus for recommending an image of an interest point, so as to determine a recommended image with a high degree of association with the interest point from a candidate image set of the interest point.
In order to achieve the above object, the following solutions are proposed:
a point-of-interest image recommendation method includes:
aiming at candidate images in the candidate image set of the interest point, obtaining image categories of the candidate images by utilizing a pre-trained classification model, and obtaining image ranking values of the candidate images by utilizing a pre-trained ranking model;
acquiring the category of the interest points;
in the candidate image set of the interest point, acquiring a candidate image corresponding to the image category with the highest category association degree of the interest point as an image to be recommended;
and determining the recommended images of the interest points according to the image sorting values of the images to be recommended.
Preferably, the pre-trained classification model and the pre-trained ranking model are two branches of a pre-trained deep learning neural network model, and the deep learning neural network model includes: the device comprises a convolution layer, at least one classification full-connection layer connected with the convolution layer, and at least one sequencing full-connection layer connected with any one classification full-connection layer;
the obtaining of the image category of the candidate image and the image ranking value of the candidate image by using the pre-trained classification model comprises:
and inputting the candidate image into the deep learning neural network model, processing the candidate image by a convolutional layer, all classification full-connection layers and all sequencing full-connection layers, obtaining the image category of the candidate image from the last classification full-connection layer, and obtaining the image sequencing value of the candidate image from the last sequencing full-connection layer.
Preferably, the method further comprises:
training a convolution layer and a classification full-link layer of the deep learning neural network model by utilizing a training image marked with an image category in advance;
after the training of the convolutional layers and the classification full-link layers is finished, setting the learning rate of the convolutional layers and the classification full-link layers to be 0;
and training the sequencing full-connection layer of the deep learning neural network model by utilizing the training images marked with the image sequencing values in advance.
Preferably, the acquiring, as an image to be recommended, a candidate image corresponding to an image category with the highest category association degree with the point of interest from the candidate image set of the point of interest includes:
searching the corresponding relation between the image category with the highest association degree of the preset interest points and the category of the interest points, and determining the image category with the highest association degree corresponding to the category of the interest points as a target image category;
and acquiring the candidate image of the target image category as an image to be recommended from the candidate image set of the interest point.
Preferably, the determining the recommended image of the interest point according to the image ranking value of the image to be recommended includes:
and selecting a preset number of images as recommended images of the interest points in the images to be recommended according to the sequence of image sorting values from high to low.
An interest point image recommendation apparatus comprising:
the image classification and sorting unit is used for obtaining the image classification of the candidate images by utilizing a pre-trained classification model aiming at the candidate images in the candidate image set of the interest points and obtaining the image sorting value of the candidate images by utilizing the pre-trained sorting model;
the interest point type obtaining unit is used for obtaining the type of the interest point;
the image acquisition unit to be recommended is used for acquiring a candidate image corresponding to the image category with the highest category relevance degree of the interest point from the candidate image set of the interest point as an image to be recommended;
and the recommended image determining unit is used for determining the recommended image of the interest point according to the image sorting value of the image to be recommended.
Preferably, the classification model and the ranking model are two branches of a deep learning neural network model, the deep learning neural network model comprising: the device comprises a convolution layer, at least one classification full-connection layer connected with the convolution layer, and at least one sequencing full-connection layer connected with any one classification full-connection layer;
the image classification and sorting unit obtains the image classification of the candidate image by using a pre-trained classification model, and obtains the image sorting value of the candidate image by using a pre-trained sorting model, and the process specifically includes:
and inputting the candidate image into the deep learning neural network model, processing the candidate image by a convolutional layer, all classification full-connection layers and all sequencing full-connection layers, obtaining the image category of the candidate image from the last classification full-connection layer, and obtaining the image sequencing value of the candidate image from the last sequencing full-connection layer.
Preferably, the method further comprises the following steps: the model training unit is used for training the deep learning neural network model, and the training process comprises the following steps:
training a convolution layer and a classification full-link layer of the deep learning neural network model by utilizing a training image marked with an image category in advance;
after the training of the convolutional layers and the classification full-link layers is finished, setting the learning rate of the convolutional layers and the classification full-link layers to be 0;
and training the sequencing full-connection layer of the deep learning neural network model by utilizing the training images marked with the image sequencing values in advance.
Preferably, the process of acquiring, by the image-to-be-recommended acquiring unit, a candidate image corresponding to an image category with the highest category association degree with the point of interest in the candidate image set of the point of interest as the image to be recommended specifically includes:
searching the corresponding relation between the image category with the highest association degree of the preset interest points and the category of the interest points, and determining the image category with the highest association degree corresponding to the category of the interest points as a target image category;
and acquiring the candidate image of the target image category as an image to be recommended from the candidate image set of the interest point.
Preferably, the process of determining the recommended image of the interest point by the recommended image determining unit according to the image ranking value of the image to be recommended specifically includes:
and selecting a preset number of images as recommended images of the interest points in the images to be recommended according to the sequence of image sorting values from high to low.
According to the technical scheme, the image recommendation scheme of the interest point obtains the image categories of the candidate images by using the classification model and the image ranking values of the candidate images by using the ranking model aiming at the candidate images in the candidate image set of the interest point, further obtains the categories of the interest point, obtains the candidate image corresponding to the image category with the highest relevance degree with the categories of the interest point in the candidate image set of the interest point as the image to be recommended, and determines the recommended image of the interest point according to the image ranking values of the image to be recommended. According to the method and the device, the relevance between the category of the candidate image and the category of the interest point and the sorting score value of the candidate image are comprehensively considered, and the recommended image with high relevance with the interest point is finally obtained.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
1 a-1 b illustrate two candidate recommended images for a point of interest;
FIG. 2 is a flowchart of an interest point image recommendation method disclosed in an embodiment of the present application;
FIG. 3 is a flowchart of a training method for a deep learning neural network model according to an embodiment of the present disclosure;
FIG. 4 illustrates a training process diagram of the ordered fully-connected layer of the model;
FIG. 5 illustrates a point of interest recommendation image obtained using the solution of the present application;
fig. 6 is a schematic diagram of a logic structure of an interest point image recommendation device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The method for recommending an interest point image disclosed in the embodiment of the present application is described with reference to fig. 2, and as shown in fig. 2, the method includes:
s100, aiming at candidate images in a candidate image set of an interest point, obtaining image categories of the candidate images by using a pre-trained classification model;
the candidate images collected through various ways and possibly taken as the recommended images of the interest points are stored in the candidate image set of the interest points. The candidate image set may include images related to or unrelated to the interest point, wherein images with low quality may also exist in the related images, and for this reason, the relevance between the image category of the candidate image and the category of the interest point is used in the present application to measure whether the candidate image can be used as a recommended image of the interest point. Therefore, this step requires first obtaining the image classification of the candidate image.
In this step, the classification model is trained in advance by using the training images marked with the image classes. The image categories may include: facia, merchandise, environment, and others. Of course, the image categories are not limited to those described in the present invention, and the image categories can be calibrated by the skilled person according to the actual needs.
Step S110, obtaining an image ranking value of the candidate image by using a pre-trained ranking model;
wherein the image ranking value indicates a degree of association of the candidate image with the point of interest. In general, a higher image ranking value indicates a higher degree of association between the candidate image and the interest point.
In this step, the ranking model is trained in advance by using the training images marked with the image ranking values. Therefore, the invention can determine the corresponding image ranking value of each candidate image in the candidate image set of the interest point through the ranking model.
Step S120, acquiring the category of the interest points;
specifically, the category of each point of interest is marked in the attribute field of the point of interest during the production process of the point of interest, so step 120 can obtain the category of the point of interest from the attribute field of the point of interest.
Step S130, in the candidate image set of the interest point, obtaining a candidate image corresponding to the image category with the highest category relevance degree of the interest point as an image to be recommended;
specifically, there are various categories of points of interest, such as "hotel", "park", "mall", and the like. For each type of interest point, the relevance to the images of different image categories is different. Therefore, the method is simple. In the invention, the relevance degree indicates the possibility that the image can be the recommended image of the interest point, and the higher the relevance degree is, the higher the possibility that the image of the image category can be the recommended image of the interest point is.
In an example case, if the category of the point of interest is "hotel", the image of the "door face" category is more highly associated with the point of interest of the "hotel" category in the four image categories of the above example, that is, the image of the "door face" category should be more used as the recommended image of the point of interest of the "hotel" category.
In this step, the category of the image with the highest relevance to the interest point may be determined as a target category according to the category of the interest point, and then a candidate image of the target image category is obtained as an image to be recommended from the candidate image set of the interest point.
In an optional implementation manner, the application may preset a correspondence between an image category with the highest relevance of the interest point and a category of the interest point, and further query the correspondence to determine the image category with the highest relevance corresponding to the category of the interest point.
It can be understood that the present application may set one-to-one, one-to-many, many-to-one or many-to-many relationship between the image category and the category of the point of interest according to actual business needs.
And step S140, determining the recommended images of the interest points according to the image sorting values of the images to be recommended.
Specifically, in step S110, the image ranking value of each candidate image in the candidate image set has been determined, and the image to be recommended is screened from the candidate image set, so the image ranking value of the image to be recommended is also determined, and in this step, the recommended image of the point of interest can be determined according to the image ranking values of the images to be recommended.
It is understood that the image ranking value indicates the degree of association between the candidate images and the interest point, and generally, the higher the image ranking value, the higher the degree of association between the candidate images and the interest point.
In practical application, in the images to be recommended, images with a preset number are selected as recommended images of the interest points according to the sequence of the image ranking values from high to low.
For example, if only an optimal recommended image of the interest point needs to be displayed, the image with the highest ranking value may be recommended as the head map, and if n (n >1) recommended images of the interest point need to be displayed, the top n images may be selected as the head map recommendation according to the image ranking value.
The method for recommending the images of the interest points, provided by the embodiment of the application, includes the steps of obtaining image categories of candidate images by using a classification model and obtaining image ranking values of the candidate images by using a ranking model respectively for the candidate images in a candidate image set of the interest points, further obtaining categories of the interest points, obtaining the candidate images corresponding to the image categories with the highest relevance degree of the categories of the interest points in the candidate image set of the interest points as images to be recommended, and determining the recommended images of the interest points according to the image ranking values of the images to be recommended. According to the method and the device, the category and the sorting score value of the candidate image are comprehensively considered, and the relevance between the recommended image of the interest point and the interest point is determined to be higher.
For the classification model and the ranking model in the above embodiments, it may be two independent deep learning neural network models, and each is trained using a corresponding training image.
In addition, the embodiment of the application also provides another optional model structure, namely a unified deep learning neural network model is designed, and the classification model and the sequencing model are used as two branches of the deep learning neural network model. Specifically, the deep learning neural network model includes:
the device comprises a convolution layer, at least one classification full-connection layer connected with the convolution layer, and at least one sequencing full-connection layer connected with any classification full-connection layer.
The convolution layer acquires the basic characteristics of the image, and the full-connection layer acquires the high-level semantics of the image. In this embodiment, the classification full-link layer is used to implement a function of classifying images, and the ranking full-link layer is used to implement a function of determining ranking score values of images. The classified full-connection layer and the sequencing full-connection layer share the convolution layer, so that the model structure is simplified, and the overall calculation speed is increased.
Based on the model provided in this embodiment, in step S100, for candidate images in the candidate image set of the point of interest, obtaining the image categories of the candidate images by using a pre-trained classification model, and in step S110, obtaining the image ranking values of the candidate images by using a pre-trained ranking model may specifically include:
and inputting the candidate image into the deep learning neural network model, processing the candidate image by a convolutional layer, all classification full-connection layers and all sequencing connection layers, obtaining the image category of the candidate image from the last classification full-connection layer, and obtaining the image sequencing value of the candidate image from the last sequencing full-connection layer.
That is, when a uniform deep learning neural network model is used, candidate images are uniformly input into a convolutional layer of the model, and the output result of the convolutional layer is processed by all the classified fully-connected layers on one hand, then the image category of the image is obtained from the last classified fully-connected layer, and on the other hand, the image ranking value of the image is obtained from the last ranked fully-connected layer after all the classified fully-connected layer is processed.
The above-described scheme is next described by way of a specific example.
In the present embodiment, the model structure used is constructed based on a cafenet network model. The ca ffenet network model is mainly composed of the first 5 convolutional layers (conv1-conv 5) and the last 3 fully-connected layers (fc6-fc 8).
In order to be applied to the present application, fc6-fc8 is used as a classified fully-connected layer in this embodiment, and a new sorting branch is added after the above-mentioned classified fully-connected layer fc6, that is, a plurality of sorted fully-connected layers are added, and the newly-added sorting branch is juxtaposed with the original fc7 layer without mutual interference. Here, the newly added sort fully-connected layer in this embodiment includes: fc7-fc 12.
On this basis, the candidate images are input into conv1, and the image categories of the candidate images are acquired from the fc8 layer, and the image ranking values of the candidate images are acquired from the fc12 layer.
In another embodiment of the present application, a training process for the above deep learning neural network model is presented. As shown in fig. 3, the training process includes:
s200, training each convolution layer and each classification full-link layer of the deep learning neural network model by utilizing a training image marked with an image class in advance;
specifically, since step S200 requires training of the convolutional layer and the classification fully-connected layer, the learning rate of the convolutional layer and the classification fully-connected layer is set to be other than 0 before training. Since the sequencing full-link layer is trained subsequently, when the convolutional layer and the classification full-link layer are trained, the learning rate of the sequencing full-link layer can be set to 0, and certainly, the learning rate can also be set to be not 0, and the training of the convolutional layer and the classification full-link layer is not influenced.
After the learning rates of the respective layers are set, the training images labeled with the image classes are used to train the respective convolution layers and the respective classification full link layers of the model in this step.
Step S210, setting the learning rate of the convolutional layer and the classification full-link layer to be 0 after the training of the convolutional layer and the classification full-link layer is finished;
specifically, when it is determined that the classification function of the model is up to standard, that is, after it is determined that training of the convolutional layer and the classification fully-connected layer is completed, training of the sequencing fully-connected layer of the model is required next, and in order to ensure that the classification stage of the model is not affected in the subsequent training process, the learning rate of the convolutional layer and the classification fully-connected layer is set to be 0, so that the subsequent training process will not cause the network parameters of the convolutional layer and the classification fully-connected layer to change, that is, the network parameters of the convolutional layer and the classification fully-connected layer will not change.
And S220, training the sequencing full-link layer of the deep learning neural network model by using the training images marked with the image sequencing values in advance.
Similar to the training process of the convolutional layer and the classification fully-connected layer, when the sequencing fully-connected layer is trained, the learning rate of the sequencing fully-connected layer is inevitably not 0, and the network parameters of the training sequencing fully-connected layer marked with the image sequencing value can be utilized.
And after the training of the sequencing full-connection layer is determined to be finished, training all network parameters in the deep learning neural network model is finished, and obtaining the trained deep learning neural network model.
In the following embodiments, the training process for ordering fully-connected layers will be described with reference to the drawings.
As shown in connection with FIG. 4, the training process of FIG. 4 is for two training images PiAnd PjThe two training images are marked with image sorting values manually, the images are respectively input into the model, and the image sorting values corresponding to the output images are obtained from the last sorting full-connection layer of the model, and are respectively as follows: f (i) and f (j).
The application may set the ordering penalty value to Σ max (0, 1-f (h) + f (n)).
Wherein f (h) is: training image PiAnd PjThe image ranking value of an image with a higher image ranking value of the middle artificial mark is predicted by a model; (n) is: training image PiAnd PjAnd the image ranking value of the image with the lower image ranking value of the middle artificial mark is predicted by the model. f (i) and f (j) are values normalized to the interval 0-1.
If the sequencing loss value is larger, the network parameter indicating the sequencing full connection layer needs to be adjusted to a larger extent. Therefore, the model is optimized more by continuously iteratively adjusting the network parameters based on the magnitude of the sequencing loss value.
Finally, the effectiveness of the scheme of the application is illustrated by a practical case. Referring to FIG. 5, a recommended image of three points of interest is illustrated.
Taking the first interest point as an example, the first interest point is "zhi dinglan" of a restaurant, the corresponding candidate image set includes 26 candidate images, and in the preset correspondence between the types of the interest points and the image types of the candidate images, the image type with the highest relevance corresponding to the restaurant is a commodity, according to the scheme of the present application, the first three images with the highest image ranking value are selected as recommended images under the category of "commodity" images, and it is obvious that the relevance between the three images and the restaurant is higher than the image with the last three ranking values shown in fig. 5.
Wherein, the numerical labels under the image in the example of fig. 5 are the sorting order of the user marks, the sorting number of 1 indicates that the user likes the most, the user's like degree decreases gradually as the number increases, and the sorting number of-1 is set uniformly for the user to be annoying. As can be seen from fig. 5, by using the scheme provided by the present application, the finally obtained recommended image not only meets the expectations of the user, but also can accurately predict the image that the user likes best as a head map recommendation.
The following describes the point of interest image recommendation device provided in the embodiment of the present application, and the point of interest image recommendation device described below and the point of interest image recommendation method described above may be referred to in correspondence with each other.
Referring to fig. 6, a schematic diagram of a logical structure of a point of interest image recommendation apparatus provided in the present application is shown, where the point of interest image recommendation apparatus includes:
the image classification and sorting unit 11 is configured to, for candidate images in a candidate image set of an interest point, obtain image categories of the candidate images by using a pre-trained classification model, and obtain image sorting values of the candidate images by using a pre-trained sorting model;
an interest point category obtaining unit 12, configured to obtain a category of an interest point;
a to-be-recommended image obtaining unit 13, configured to obtain, in the candidate image set of the interest point, a candidate image corresponding to an image category with the highest category relevance degree to the interest point as a to-be-recommended image;
and the recommended image determining unit 14 is configured to determine a recommended image of the interest point according to the image ranking value of the image to be recommended.
The device for recommending the images of the interest points, provided by the embodiment of the application, respectively obtains the image categories of the candidate images by using the classification model and the image ranking values of the candidate images by using the ranking model for the candidate images in the candidate image set of the interest points, further obtains the categories of the interest points, obtains the candidate images corresponding to the image category with the highest relevance degree to the categories of the interest points in the candidate image set of the interest points as images to be recommended, and determines the recommended images of the interest points according to the image ranking values of the images to be recommended. According to the method and the device, the category and the sorting score value of the candidate image are comprehensively considered, and the relevance between the recommended image of the interest point and the interest point is determined to be higher.
Optionally, the classification model and the ranking model may be two branches of a deep learning neural network model, the deep learning neural network model including: the device comprises a convolution layer, at least one classification full-connection layer connected with the convolution layer, and at least one sequencing full-connection layer connected with any one classification full-connection layer;
the image classifying and sorting unit obtains the image category of the candidate image by using a pre-trained classification model, and obtains the image sorting value of the candidate image by using a pre-trained sorting model, which may specifically include:
and inputting the candidate image into the deep learning neural network model, processing the candidate image by a convolutional layer, all classification full-connection layers and all sequencing full-connection layers, obtaining the image category of the candidate image from the last classification full-connection layer, and obtaining the image sequencing value of the candidate image from the last sequencing full-connection layer.
Optionally, the apparatus of the present application may further include: a model training unit, configured to train the deep learning neural network model, where the training process may include:
training a convolution layer and a classification full-link layer of the deep learning neural network model by utilizing a training image marked with an image category in advance;
after the training of the convolutional layers and the classification full-link layers is finished, setting the learning rate of the convolutional layers and the classification full-link layers to be 0;
and training the sequencing full-connection layer of the deep learning neural network model by utilizing the training images marked with the image sequencing values in advance.
Optionally, the process of acquiring, by the image-to-be-recommended acquiring unit, a candidate image corresponding to an image category with the highest category association degree with the point of interest in the candidate image set of the point of interest as the image to be recommended may specifically include:
searching the corresponding relation between the image category with the highest association degree of the preset interest points and the category of the interest points, and determining the image category with the highest association degree corresponding to the category of the interest points as a target image category;
and acquiring the candidate image of the target image category as an image to be recommended from the candidate image set of the interest point.
Optionally, the process of determining the recommended image of the interest point by the recommended image determining unit according to the image ranking value of the image to be recommended may specifically include:
and selecting a preset number of images as recommended images of the interest points in the images to be recommended according to the sequence of image sorting values from high to low.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A point-of-interest image recommendation method is characterized by comprising the following steps:
aiming at candidate images in the candidate image set of the interest point, obtaining image categories of the candidate images by utilizing a pre-trained classification model, and obtaining image ranking values of the candidate images by utilizing a pre-trained ranking model;
acquiring the category of the interest points;
in the candidate image set of the interest point, acquiring a candidate image corresponding to the image category with the highest category association degree of the interest point as an image to be recommended;
and determining the recommended images of the interest points according to the image sorting values of the images to be recommended.
2. The method of claim 1, wherein the pre-trained classification model and the pre-trained ranking model are two branches of a pre-trained deep learning neural network model, the deep learning neural network model comprising: the device comprises a convolution layer, at least one classification full-connection layer connected with the convolution layer, and at least one sequencing full-connection layer connected with any one classification full-connection layer;
the obtaining of the image category of the candidate image and the image ranking value of the candidate image by using the pre-trained classification model comprises:
and inputting the candidate image into the deep learning neural network model, processing the candidate image by a convolutional layer, all classification full-connection layers and all sequencing full-connection layers, obtaining the image category of the candidate image from the last classification full-connection layer, and obtaining the image sequencing value of the candidate image from the last sequencing full-connection layer.
3. The method of claim 2, further comprising:
training a convolution layer and a classification full-link layer of the deep learning neural network model by utilizing a training image marked with an image category in advance;
after the training of the convolutional layers and the classification full-link layers is finished, setting the learning rate of the convolutional layers and the classification full-link layers to be 0;
and training the sequencing full-connection layer of the deep learning neural network model by utilizing the training images marked with the image sequencing values in advance.
4. The method according to any one of claims 1 to 3, wherein the obtaining, as the image to be recommended, a candidate image corresponding to an image category with the highest category relevance degree to the point of interest from the candidate image set of the point of interest includes:
searching the corresponding relation between the image category with the highest association degree of the preset interest points and the category of the interest points, and determining the image category with the highest association degree corresponding to the category of the interest points as a target image category;
and acquiring the candidate image of the target image category as an image to be recommended from the candidate image set of the interest point.
5. The method according to any one of claims 1 to 3, wherein the determining the recommended image of the interest point according to the image ranking value of the image to be recommended comprises:
and selecting a preset number of images as recommended images of the interest points in the images to be recommended according to the sequence of image sorting values from high to low.
6. A point-of-interest image recommendation apparatus, comprising:
the image classification and sorting unit is used for obtaining the image classification of the candidate images by utilizing a pre-trained classification model aiming at the candidate images in the candidate image set of the interest points and obtaining the image sorting value of the candidate images by utilizing the pre-trained sorting model;
the interest point type obtaining unit is used for obtaining the type of the interest point;
the image acquisition unit to be recommended is used for acquiring a candidate image corresponding to the image category with the highest category relevance degree of the interest point from the candidate image set of the interest point as an image to be recommended;
and the recommended image determining unit is used for determining the recommended image of the interest point according to the image sorting value of the image to be recommended.
7. The apparatus of claim 6, wherein the classification model and the ranking model are two branches of a deep learning neural network model, the deep learning neural network model comprising: the device comprises a convolution layer, at least one classification full-connection layer connected with the convolution layer, and at least one sequencing full-connection layer connected with any one classification full-connection layer;
the image classification and sorting unit obtains the image classification of the candidate image by using a pre-trained classification model, and obtains the image sorting value of the candidate image by using a pre-trained sorting model, and the process specifically includes:
and inputting the candidate image into the deep learning neural network model, processing the candidate image by a convolutional layer, all classification full-connection layers and all sequencing full-connection layers, obtaining the image category of the candidate image from the last classification full-connection layer, and obtaining the image sequencing value of the candidate image from the last sequencing full-connection layer.
8. The apparatus of claim 7, further comprising: the model training unit is used for training the deep learning neural network model, and the training process comprises the following steps:
training a convolution layer and a classification full-link layer of the deep learning neural network model by utilizing a training image marked with an image category in advance;
after the training of the convolutional layers and the classification full-link layers is finished, setting the learning rate of the convolutional layers and the classification full-link layers to be 0;
and training the sequencing full-connection layer of the deep learning neural network model by utilizing the training images marked with the image sequencing values in advance.
9. The apparatus according to any one of claims 6 to 8, wherein the process of acquiring, by the image to be recommended acquisition unit, a candidate image corresponding to an image category with the highest category association degree with the point of interest from the candidate image set of the point of interest as the image to be recommended specifically includes:
searching the corresponding relation between the image category with the highest association degree of the preset interest points and the category of the interest points, and determining the image category with the highest association degree corresponding to the category of the interest points as a target image category;
and acquiring the candidate image of the target image category as an image to be recommended from the candidate image set of the interest point.
10. The apparatus according to any one of claims 6 to 8, wherein the process of determining the recommended image of the point of interest by the recommended image determining unit according to the image ranking value of the image to be recommended specifically includes:
and selecting a preset number of images as recommended images of the interest points in the images to be recommended according to the sequence of image sorting values from high to low.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810111415.4A CN110209916B (en) | 2018-02-05 | 2018-02-05 | Method and device for recommending point of interest images |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810111415.4A CN110209916B (en) | 2018-02-05 | 2018-02-05 | Method and device for recommending point of interest images |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110209916A CN110209916A (en) | 2019-09-06 |
| CN110209916B true CN110209916B (en) | 2021-08-20 |
Family
ID=67778557
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810111415.4A Active CN110209916B (en) | 2018-02-05 | 2018-02-05 | Method and device for recommending point of interest images |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110209916B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027600B (en) * | 2019-11-25 | 2021-03-23 | 腾讯科技（深圳）有限公司 | Image category prediction method and device |
| CN111353851A (en) * | 2020-02-27 | 2020-06-30 | 携程计算机技术（上海）有限公司 | Hotel sorting recommendation method and device, electronic equipment and storage medium |
| CN111461228B (en) * | 2020-04-01 | 2024-03-15 | 腾讯科技（深圳）有限公司 | Image recommendation method and device and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102594905A (en) * | 2012-03-07 | 2012-07-18 | 南京邮电大学 | Method for recommending social network position interest points based on scene |
| CN103258022A (en) * | 2013-05-07 | 2013-08-21 | 天津大学 | Local commerce service recommendation system and method based on user interest |
| CN104537027A (en) * | 2014-12-19 | 2015-04-22 | 百度在线网络技术（北京）有限公司 | Information recommendation method and device |
| CN105681830A (en) * | 2016-02-06 | 2016-06-15 | 优住（北京）科技股份公司 | Advertisement playing system and playing platform based on interest points of user |
| WO2016180182A1 (en) * | 2015-10-30 | 2016-11-17 | 中兴通讯股份有限公司 | Service package recommendation method and device |
| CN107239203A (en) * | 2016-03-29 | 2017-10-10 | 北京三星通信技术研究有限公司 | A kind of image management method and device |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101170882B1 (en) * | 2010-08-12 | 2012-08-03 | 팅크웨어(주) | Apparatus and Method for Displaying Point of Interest |
| KR20170045919A (en) * | 2015-10-20 | 2017-04-28 | 주식회사 카카오 | Method, server and computer readable recording medium for dynamically wrapping poi image in three dimensional map |
| CN107560610A (en) * | 2016-06-30 | 2018-01-09 | 上海博泰悦臻网络技术服务有限公司 | A kind of electronic equipment and its method for information display and system of application |
| CN107560608B (en) * | 2016-06-30 | 2021-06-04 | 上海博泰悦臻网络技术服务有限公司 | Electronic equipment and information display method and system applied by same |
| CN107330750B (en) * | 2017-05-26 | 2019-03-08 | 北京三快在线科技有限公司 | A kind of recommended products figure method and device, electronic equipment |
-
2018
- 2018-02-05 CN CN201810111415.4A patent/CN110209916B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102594905A (en) * | 2012-03-07 | 2012-07-18 | 南京邮电大学 | Method for recommending social network position interest points based on scene |
| CN103258022A (en) * | 2013-05-07 | 2013-08-21 | 天津大学 | Local commerce service recommendation system and method based on user interest |
| CN104537027A (en) * | 2014-12-19 | 2015-04-22 | 百度在线网络技术（北京）有限公司 | Information recommendation method and device |
| WO2016180182A1 (en) * | 2015-10-30 | 2016-11-17 | 中兴通讯股份有限公司 | Service package recommendation method and device |
| CN105681830A (en) * | 2016-02-06 | 2016-06-15 | 优住（北京）科技股份公司 | Advertisement playing system and playing platform based on interest points of user |
| CN107239203A (en) * | 2016-03-29 | 2017-10-10 | 北京三星通信技术研究有限公司 | A kind of image management method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110209916A (en) | 2019-09-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108804633B (en) | Content recommendation method based on behavior semantic knowledge network | |
| CN108256568B (en) | Plant species identification method and device | |
| CN106355449B (en) | User selection method and device | |
| KR101579376B1 (en) | Personalized place recommendation system and method by using subjectivity analysis for user classification | |
| CN109189991A (en) | Repeat video frequency identifying method, device, terminal and computer readable storage medium | |
| KR20190070911A (en) | How to recommend an instructor in an online lecture system | |
| CN109978812A (en) | Camera system, learning device, photographic device and learning method | |
| WO2019015246A1 (en) | Image feature acquisition | |
| CN110209916B (en) | Method and device for recommending point of interest images | |
| US10838985B2 (en) | Item recommendation method, device, and system | |
| KR101424382B1 (en) | Method for recommending point of interest using user preferences and moving patterns | |
| CN109597858B (en) | Merchant classification method and device and merchant recommendation method and device | |
| CN109740072B (en) | Hotel ordering method and system based on POI under OTA platform | |
| CN107391577B (en) | Work label recommendation method and system based on expression vector | |
| CN110992124A (en) | House resource recommendation method and system | |
| CN109408710A (en) | Search result optimization method, device, system and storage medium | |
| CN109741108A (en) | Streaming application recommended method, device and electronic equipment based on context aware | |
| CN110765348B (en) | Hot word recommendation method and device, electronic equipment and storage medium | |
| CN112765374A (en) | Education resource screening system and method for information push | |
| CN110515929B (en) | Book display method, computing device and storage medium | |
| CN112699311A (en) | Information pushing method, storage medium and electronic equipment | |
| CN111782978A (en) | Method and device for processing point of interest data, electronic equipment and readable medium | |
| CN110705889A (en) | Enterprise screening method, device, equipment and storage medium | |
| CN111414966B (en) | Classification method, classification device, electronic equipment and computer storage medium | |
| CN114969135A (en) | Personalized tour route recommendation method, device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200421 Address after: 310052 room 508, floor 5, building 4, No. 699, Wangshang Road, Changhe street, Binjiang District, Hangzhou City, Zhejiang Province Applicant after: Alibaba (China) Co.,Ltd. Address before: 102200, No. 18, No., Changsheng Road, Changping District science and Technology Park, Beijing, China. 1-5 Applicant before: AUTONAVI SOFTWARE Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |