JP7469452B1 - RECOMMENDATION DECISION DEVICE, RECOMMENDATION DECISION METHOD, AND PROGRAM - Google Patents

Abstract

The present invention provides a recommendation determination device, a recommendation determination method, and a program that can associate a theme with an item to be recommended without manual intervention.
[Solution] A recommendation determination device that determines recommended items to be recommended to a user of items provided by electronic means, the recommendation determination device comprising: a theme estimation unit that estimates a theme of interest, which is the theme of the recommended items to be presented to the user as a recommendation, based on the user's item selection history and themes associated with the items; an item estimation unit that estimates the importance of each item in each of the themes of interest estimated by the theme estimation unit; and a recommendation determination unit that determines recommended items for the user for each of the themes of interest based on the ranking results of the importance.
[Selected Figure] Figure 1

Description

The present invention relates to a recommendation determination device, a recommendation determination method, and a program.

Conventionally, a technology has been proposed that estimates a consumer's interests and provides advertisements related to the estimated interests in a way that is easy for the consumer to accept, effectively providing the advertisements (see, for example, Patent Document 1). Patent Document 1 describes estimating the level of preference of what a consumer is interested in and to what extent based on information related to the consumer, selectively providing advertisements related to themes specified by the subject estimated to be of interest or a category corresponding to the subject, and changing the display mode of the advertisements related to the above-mentioned subject depending on the estimated level of preference.

JP 2017-207936 A

However, with conventional technology, matching themes with recommended items (e.g., advertisements) was done manually, which was inefficient and burdensome.

The present invention has been made in consideration of these circumstances, and one of its objectives is to provide a recommendation determination device, recommendation determination method, and program that can associate themes with recommended items (e.g., advertisements) without human intervention.

One aspect of the present invention is a recommendation determination device that determines recommended items to be recommended to a user of items provided by electronic means, the recommendation determination device including: a theme estimation unit that estimates a theme of interest, which is the theme of the recommended items to be presented as a recommendation to the user, based on the user's item selection history and themes associated with the items; an item estimation unit that estimates the importance of each item in each of the themes of interest estimated by the theme estimation unit; and a recommendation determination unit that determines recommended items for the user for each of the themes of interest based on the ranking results of the importance.

According to one aspect of the present invention, it is possible to provide a recommendation determination device, a recommendation determination method, and a program that can associate a theme with a recommended item (e.g., an advertisement) without human intervention.

1 is a diagram illustrating an example of a system configuration of an information processing system 1 according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of the configuration of a content server 100. FIG. 11 is a diagram showing an example of the flow of pre-processing executed by the content server 100. FIG. 1 is a first diagram showing an outline of a first pre-treatment; FIG. 2 is a second diagram showing an outline of the first pretreatment. FIG. 1 is a first diagram showing an overview of a second pre-treatment. FIG. 2 is a second diagram showing an outline of the second pre-treatment. FIG. 11 is a diagram illustrating an example of the flow of a content distribution process. FIG. 13 is a diagram illustrating an example of a sampling period and an effect of reranking.

[overview]
Hereinafter, with reference to the drawings, embodiments of the recommendation determination device, recommendation determination method, and program of the present invention will be described. The recommendation determination device is realized by one or more processors. The recommendation determination device may be a device that provides cloud services, or a device in which a program such as a tool or firmware is installed and capable of executing processing by itself. In other words, there are no particular restrictions on the computer device for realizing the recommendation determination device as long as it is capable of distributing content to the user's terminal device, and the recommendation determination device may be realized by any computer device that can execute the processing described below.

[composition]
FIG. 1 is a diagram showing an example of the system configuration of the information processing system 1 of this embodiment. The information processing system 1 is a system that provides a user with content related to a specific group of items provided by electronic means. The group of items is not limited to a collection of items of a specific type, but in this embodiment, a case where the group of items is a collection of books will be described. In this case, the information processing system 1 provides the user with information related to the books included in the group of items as content. The information processing system 1 in this case may be, for example, a book sales system or a book browsing system. Here, the information processing system 1 provides the user with a list of books included in the group of items, accepts the selection of a book desired by the user from the provided list, and provides the user with content related to the selected book. The list of books provided to the user may be a list of all books included in the group of items, or may be a list of books classified from various viewpoints. Furthermore, the information processing system 1 has a function of selecting a recommended book (hereinafter also referred to as a "recommendation") related to the selected book in response to the selection of any book by the user and presenting it to the user.

The information processing system 1 includes, for example, a content server 100, a database server 200, and a user terminal device 300. The content server 100 is a server that provides the user with content related to books included in an item group. In providing content, the content server 100 provides the user with a list of the above-mentioned books and accepts the selection of a book from the list. The content server 100 selects books to be recommended for the selected book (hereinafter referred to as "recommended books") and presents them to the user. The content server 100 obtains information necessary for providing content and information necessary for selecting recommendations from the database server 200. The content server 100 is an example of a "recommendation determination device" in the present invention.

The database server 200 is a server that stores and manages information necessary for content distribution by the content server 100. In response to a request from the content server 100, the database server 200 extracts the requested information from the information it holds and provides it to the content server 100. In addition, in response to a request from the content server 100, the database server 200 performs update processing such as addition, modification, and deletion on the information it holds. In this embodiment, the database server 200 is described as being configured separately from the content server 100, but it may also be configured integrally with the content server 100.

The user terminal device 300 is an information processing terminal used by the user when receiving the content provision service from the content server 100. For example, the user terminal device 300 is a communication terminal such as a PC, a smartphone, or a tablet. The user terminal device 300 acquires a list of books from the content server 100 and displays it, and also accepts an operation to select a book from the list and notifies the content server 100 of the selected book. The user terminal device 300 acquires from the content server 100 and displays a list of books recommended for the book, along with information about the notified book.

The content server 100, the database server 200, and the user terminal device 300 can communicate with each other via a communication network NW. The communication network NW is, for example, the Internet, a mobile communication network, a LAN (Local Area Network), or a WAN (Wide Area Network). For example, the LAN may be a wired LAN (Local Area Network) of a specific standard such as Ethernet, or a wireless LAN of various standards such as Wi-Fi and Bluetooth (registered trademark). The communication network NW may be a transmission path for wired communication, a transmission path for wireless communication, or a combination of a transmission path for wireless communication and a transmission path for wired communication.

Figure 2 is a diagram showing an example of the configuration of the content server 100. The content server 100 includes, for example, a communication unit 110, a content distribution unit 120, a preprocessing unit 130, a recommendation determination unit 140, a display priority determination unit 150, and a storage unit 160. These components are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device such as a HDD (Hard Disk Drive) or a flash memory, or may be stored in a removable storage medium such as a DVD or a CD-ROM, and may be installed by mounting the storage medium in a drive device.

The communication unit 110 is a communication interface that connects the content server 100 to the communication network NW. The content server 100 can communicate with the database server 200 and the user terminal device 300 via the communication unit 110.

The content distribution unit 120 provides the user with content related to books in the item group. In response to a request from the user terminal device 300, the content distribution unit 120 generates distribution data for content related to the book specified by the user and transmits it to the user terminal device 300. For example, the content distribution unit 120 creates distribution data based on content information 161 (described later) stored in the storage unit 160. The content information 161 includes various information required to generate distribution data for the content. For example, the content information 161 includes item information and image data related to the item group. The content information 161 is stored in the storage unit 160 in advance of the content distribution.

The pre-processing unit 130 executes pre-processing to generate information necessary for the recommendation determination unit 140 to execute processing to determine recommended books. More specifically, the pre-processing unit 130 includes a theme estimation unit 132 and a book estimation unit 134. The theme estimation unit 132 executes a first pre-processing to estimate themes of high interest to the user (hereinafter referred to as "interest themes") and determine the priority of recommendations for each estimated interest theme. The book estimation unit 134 executes a second pre-processing to determine recommended books to be presented to the user for each theme estimated as the user's interest theme in the first pre-processing. The book estimation unit 134 is an example of an "item estimation unit".

As a result of the first preprocessing, the preprocessing unit 130 generates a recommendation list 162 indicating the priority of recommendations for each interest theme and registers it in the database server 200. As a result of the second preprocessing, the preprocessing unit 130 generates a theme book list 163 indicating the correspondence between each interest theme and recommended books and registers it in the database server 200. Note that the preprocessing unit 130 may be provided in a device separate from the content server 100. In this case, the results of the preprocessing may be provided to the content server 100 via the communication network NW or the database server 200.

The recommendation determination unit 140 determines the recommended books to be presented to the user. More specifically, the recommendation determination unit 140 refers to the recommendation list 162 to recognize the themes of interest to be recommended to the user, and refers to the thematic book list 163 to determine the recommended books for each theme of interest.

The display priority determination unit 150 determines the display priority on the user terminal device 300 for the theme of interest determined by the recommendation determination unit 140. The display priority determination unit 150 notifies the content distribution unit 120 of the determined display priority of the theme of interest. The content distribution unit 120 generates content data so that content related to the theme of interest is displayed with the notified display priority, and distributes the content data to the user terminal device 300.

The storage unit 160 is realized by a storage device such as a HDD, flash memory, or RAM (Random Access Memory). The storage unit 160 stores content information 161 required for content distribution, temporary data, programs, program setting information, and the like. The storage unit 160 may also be used as a storage area for a recommendation list 162, a theme book list 163, and the like downloaded from the database server 200.

Figure 3 is a diagram showing an example of the flow of preprocessing executed by the content server 100. First, in the content server 100, the preprocessing unit 130 executes the first preprocessing to generate a recommendation list 162 for each user (step S101). For example, the preprocessing unit 130 obtains a ratio indicating the extent to which the user has come into contact with each theme (hereinafter referred to as "contact rate"), and learns a theme estimation model for estimating the user's interest theme based on the user's book selection results by machine learning using the user's contact rate for each theme as training data. Note that "contact" here means that the user selects a theme. In this embodiment, browsing (selecting) a book associated with a theme is considered to be contact with the theme. For example, the preprocessing unit 130 recognizes the themes that the user has come into contact with and the number of contacts with each theme based on the user's book browsing history, and calculates the ratio of the number of contacts with each theme to the number of contacts with all themes as the contact rate. The preprocessing unit 130 calculates the contact rate for each user and for each theme.

Next, the preprocessing unit 130 performs machine learning using the training data labeled with the calculated contact rate to learn the relationship between the user's contact rate for the theme and the combination of the user and the theme.

Figure 4 is a first diagram showing an overview of the first preprocessing. For example, the preprocessing unit 130 uses this machine learning to learn a theme estimation model M1 that inputs information INF1 about the characteristics of a user and a combination of themes to be learned, and outputs information INF2 indicating the proportion of exposure (exposure rate) of the user to the theme. The user characteristics may include any item that is considered to have a high probability of being correlated with the selection of the theme, such as the user's gender, age, and area of residence. Theme characteristics may include the book title, genre, publisher, author, etc. For example, a lightGBM (Light Gradient Boosting Machine) can be used to train the theme estimation model.

The preprocessing unit 130 then creates a recommendation list 162 using the learned theme estimation model. FIG. 5 is a second diagram showing an overview of the first preprocessing. Specifically, the preprocessing unit 130 inputs the user and theme characteristics for all combinations of all users and all themes in the information processing system 1 into the theme estimation model M1, and obtains, as the output, information INF3 on the contact rate for each theme for each user. The preprocessing unit 130 determines those themes with a contact rate equal to or greater than a predetermined threshold as the user's theme of interest, and registers the themes of interest associated with the identification information of each user in the database server 200 as the recommendation list 162.

Returning to the explanation of FIG. 3, after the first pre-processing is completed, the pre-processing unit 130 executes the second pre-processing to generate a theme book list 163 for each user's theme of interest (step S102). For example, the pre-processing unit 130 executes a preference estimation process for the theme of interest to estimate the strength of the user's preference for the theme of interest (hereinafter referred to as "preference level"). For example, the preference estimation process is realized by executing the following steps (1) to (3) for each user.

(1) The preprocessing unit 130 calculates the number of times the user has come into contact with a theme that the user has actually come into contact with based on the user's browsing history. The number of times the user has come into contact can be calculated, for example, by counting the number of times an associated theme appears for each book that the user has viewed.

(2) The preprocessing unit 130 scores the number of contacts with each theme obtained in (1) using the Tf-Idf (term frequency - inverse document frequency) method. Tf-Idf is a statistical measure that indicates "how important each term in each document is in that document." Specifically, it is the product of the Tf (term frequency) value, which indicates "how frequently a certain word appears in a certain document," and the Idf (inverse document frequency) value, which indicates "(conversely) how infrequently" a "document containing a certain word appears in all documents." Here, documents correspond to users, and words correspond to themes.

(3) The preprocessing unit 130 weights the scores for each theme obtained in (2) using the Softmax function and outputs the weighted scores as the user's preference for each theme. FIG. 6 is a first diagram showing an outline of the second preprocessing. For example, in the example of FIG. 6, the preprocessing unit 130 obtains an Idf value for user #1 using information INF4 on the number of contacts of all users with theme A, obtains a Tf value for user #1 using information INF5 on the number of contacts of all themes with user #1, and obtains a Tf-Idf value for user #1 using the obtained Tf and Idf values. The Tf-Idf value obtained in this manner indicates the importance of theme A to user #1, and in this embodiment, this is defined as user #1's preference for theme A.

The preprocessing unit 130 then performs a basis theme estimation process based on the user's estimated preference for the interest theme, thereby estimating which theme was the basis for the user's viewing of the book. Hereinafter, when a user views a book based on interest in a certain theme, the theme will be referred to as the "basis theme" as the theme that was the basis for the user's viewing of the book. For example, the basis theme estimation process is realized by executing the following steps (4) and (5) for each user.

(4) The preprocessing unit 130 scores the number of times a book has been viewed for a combination of a book and a theme associated with the book, taking into account the user's preference for the theme, and obtains the number of times the book has been viewed based on interest in the theme (hereinafter referred to as the "basis count"). For example, the preprocessing unit 130 calculates the number of times a book has been viewed by a user multiplied by the user's preference for the theme for all users, and adds up the result for all users to obtain the basis count for the combination of the book and the theme.

(5) The preprocessing unit 130 scores the number of times each book was viewed based on interest in each theme using the Tf-Idf method. Here, documents correspond to themes, and words correspond to books. FIG. 7 is a second diagram showing an outline of the second preprocessing. For example, in the example of FIG. 7, the preprocessing unit 130 acquires information INF7 indicating the total number of times each user viewed a book based on interest in a theme (number of times of basis) for each book based on information indicating the preference of each user and the book viewing history INF6 by each user. Note that the example of FIG. 7 shows a case where the recommendation list 162 created in the first preprocessing is used as information indicating the preference of each user, but a preference estimated by a process or method other than the first preprocessing may be used instead of the recommendation list 162.

Then, the preprocessing unit 130 obtains an Idf value for book a using information INF71 indicating the number of grounds for all themes for book a from information INF7 of the number of grounds by all users, and obtains a Tf value for theme A using information INF72 indicating the number of grounds for theme A in all books, and obtains a Tf-Idf value for theme A using the obtained Tf value and Idf value. The Tf-Idf value obtained in this way is a single score common to all users that indicates the importance of book a in theme A. The preprocessing unit 130 ranks the scores of the theme among the books associated with the theme, and determines the top books as recommended books for the theme to be presented to the user. The preprocessing unit 130 generates a theme book list 163 that lists the recommended books determined for each of the user's interest themes, and registers it in the database server 200.

Figure 8 is a diagram showing an example of the flow of a process (hereinafter referred to as "content distribution process") in which the content server 100 distributes content. For example, the process flow in Figure 8 is initiated when the content distribution unit 120 in the content server 100 receives a content distribution request from the user terminal device 300. The content distribution request is transmitted from the user terminal device 300 to the content server 100 in response to, for example, the user inputting an operation to select a specific book into the user terminal device 300.

First, based on the content distribution request received from the user terminal device 300, the content distribution unit 120 recognizes the requesting user (hereinafter referred to as the "target user") and the book selected by the target user (hereinafter referred to as the "target book") (step S201), and instructs the recommendation determination unit 140 to determine a recommended book to be presented to the target user in response to the selection of the target book.

The recommendation determination unit 140 estimates the interest theme of the target user based on the combination of the target user and the target book (step S202). For example, the recommendation determination unit 140 obtains from the database server 200 a list of interest themes corresponding to the combination of the target user and the target book from the recommendation list 162 created in advance by preprocessing. For example, the recommendation list 162 is generated or updated by daily batch processing and registered in the database server 200. Note that the content server 100 may have a configuration in which a theme estimation model generated or updated by preprocessing is stored in the storage unit 160 instead of a configuration in which the database server 200 is inquired about the interest theme. In this case, the recommendation determination unit 140 may rank the contact rates of each theme obtained by inputting the characteristics of the target user and the target book into the theme estimation model, and determine the top theme as the interest theme.

Then, the recommendation determination unit 140 determines recommended books to be presented to the target user for the theme of interest estimated in step S202 (step S203). For example, the recommendation determination unit 140 obtains from the database server 200 a list of books corresponding to the theme of interest from the theme book list 163 created in advance by preprocessing. For example, the theme book list 163 is generated or updated by daily batch processing and registered in the database server 200.

Then, the display priority determination unit 150 determines the display priority in the user terminal device 300 for the topic of interest determined by the recommendation determination unit 140 (step S204). For example, the display priority determination unit 150 determines the display priority of the topic of interest using a multi-armed bandit algorithm such as Thompson Sampling. More specifically, the display priority determination unit 150 can determine the display priority by executing the following steps (6) and (7).

(6) The display priority determination unit 150 samples the number of times the user has come into contact with each theme by Thompson sampling based on the book browsing history of the target user, and defines a probability density function assuming a Bernoulli distribution for the sampling results. The display priority determination unit 150 applies random numbers to the defined probability density function to calculate a score (hereinafter referred to as the "first score") for reflecting the target user's relatively recent contact status with each theme in the display priority. Note that the extent to which the contact status is taken into account may be adjusted by the sampling period for the number of contacts. The sampling period may be adjusted as appropriate depending on the type of item (here, books) and the method of providing the content.

(7) The display priority determination unit 150 determines the display priority of the interest theme based on the second score S2 obtained by applying the first score S1 determined in (6) to the target user's preference F for the interest theme. For example, if the level of preference indicates the level of interest in the interest theme, the higher the preference, the higher the priority of each theme. In this case, the display priority determination unit 150 calculates the second score S2 using the following formulas (1) and (2).
S2(U,T)=F(U,T)×S1(U,T) (1)
F(U, T)=1/r (2)

In formula (1), S1(U, T) is the first score obtained for each theme T of the themes of interest. The first score S1(U, T) is obtained by sampling the number of times the target user U has come into contact with theme T based on the browsing history of the target user U. F(U, T) is the preference of the target user U for theme T. For example, F(U, T) can be the reciprocal of the priority r of theme T that is prioritized for the target user U. The display priority determination unit 150 determines the display priority of the themes of interest by ranking the second scores S2(U, T) obtained for each theme for the target user U.

By determining the display priority in this manner, it is possible to prevent the display priority of a topic of interest from becoming fixed. As a result, it is possible to appropriately change the recommended books presented to the user in accordance with the user's tendency for interest in the topic to change. Hereinafter, appropriately changing the display priority of a topic in this manner in accordance with the user's exposure to the topic will be referred to as "reranking" the display priority. "Reranking" is an example of correcting preference levels.

Figure 9 shows an example of the sampling period and the effect of reranking. The first graph on the upper side of Figure 9 shows the probability density function when the sampling period is 30 days, and the second graph on the lower side of Figure 9 shows the probability density function when the sampling period is 180 days. In both the first and second graphs, the vertical axis represents the number of contacts with a theme, and the horizontal axis represents the probability of the corresponding number of contacts occurring. The first graph has a larger spread along the horizontal axis of the probability density function than the second graph. In other words, when random numbers are generated according to a probability density function (the range of the domain in the beta distribution is 0 to 1), the variance of S1(T) is larger in the case of the first graph, and reranking may have an excessive effect compared to the case of the second graph.

On the other hand, the second graph has a smaller spread along the horizontal axis of the probability density function than the first graph. In other words, when random numbers are generated according to a probability density function, the variance of S1(T) is smaller in the case of the second graph, and the effect of reranking is suppressed compared to the case of the first graph. For this reason, if a user's contact history is not sufficiently accumulated, the strength of the effect of reranking can be adjusted by expanding the sample period, such as by padding the sample data. The example in Figure 9 is an example in which a sampling period of 30 days is multiplied by 6 to pad it to 180 days.

Returning to the explanation of FIG. 8, the content distribution unit 120 then generates content data in which recommended books on the topic of interest are displayed with the display priority determined by the display priority determination unit 150, and distributes the content data to the requesting user terminal device 300 (step S205).

According to the information processing system 1 of the embodiment configured as described above, it is possible to provide more appropriate recommendations to users without manually matching themes with recommended books. Furthermore, according to the information processing system 1 of the embodiment, in addition to not requiring manual work for the above matching, it is possible to perform matching with high accuracy.

<Modification>
The content server 100 and the database server 200 may be configured as separate devices capable of communicating with each other via the network NW as in the above embodiment, or may be configured as an integrated unit.

In addition, the content distribution function of the content server 100, the function of determining recommended books in response to a user's browsing (selection) of books, and the function of performing pre-processing for determining recommended books may each be implemented in separate devices, or two or more of the functions may be implemented in the same device.

The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made without departing from the spirit of the present invention.

1... Information processing system, 100... Content server, 110... Communication unit, 120... Content distribution unit, 130... Preprocessing unit, 132... Theme estimation unit, 134... Book estimation unit, 140... Recommendation determination unit, 150... Display priority determination unit, 160... Storage unit, 161... Content information, 162... Recommendation list, 163... Theme book list, 200... Database server, 300... User terminal device

Claims (8)

A recommendation determination device that determines a recommended item to be recommended to a user of an item provided by an electronic means, comprising:
a theme estimation unit that estimates a theme of interest, which is a theme of a recommended item to be presented as a recommendation to the user, based on a selection history of items by the user and a theme associated with the items;
an item estimation unit that estimates the importance of each item in each of the themes of interest estimated by the theme estimation unit;
a recommendation determination unit that determines a recommended item for the user for each of the interest themes based on the ranking result of the importance;
A recommendation determination device comprising: The theme estimation unit learns a theme estimation model using a machine learning technique, in which the user and the background of the theme are input and a contact rate, which is a rate at which the user selects each theme, is output, and determines a theme having a contact rate equal to or higher than a threshold value as a theme of interest to the user.
The recommendation determination device according to claim 1 . The theme estimation unit calculates a first Tf-Idf value when a number of contacts, which is the number of times a user has selected an item associated with each theme, is defined as a document and a theme is defined as a word, and estimates the user's preference for each theme based on the first Tf-Idf value.
The recommendation determination device according to claim 1 . the item estimation unit estimates a basis count, which is the number of times the user selected an item on the basis of an interest in a certain theme, based on the user's preference for each theme and the user's selection history of the items, calculates a second Tf-Idf value in the case where the theme is a document and the item is a word, based on the basis count for each combination of theme and item, and estimates the importance of each item in each theme for the user based on the second Tf-Idf value;
The recommendation determination device according to claim 3 . A priority determination unit that determines a priority of a recommendation for each of the themes estimated as the theme of interest,
the priority determination unit determines the priority of the themes of interest based on the degree of preference estimated for each theme.
The recommendation determination device according to claim 3 . the priority determination unit corrects the preference degree used when determining the priority according to a selection state of the items selected by the user during a predetermined period in the past.
The recommendation determination device according to claim 5 . A recommendation determination device that determines a recommended item to be recommended to a user of an item provided by an electronic means,
estimating a theme of interest, which is a theme of a recommended item to be presented as a recommendation to the user, based on a selection history of items by the user and a theme associated with the items;
For each of the estimated themes of interest, estimating the importance of each item in the theme;
determining a recommended item for the user for each of the themes of interest based on the ranking result of the importance;
How recommendations are determined. A recommendation determination device that determines a recommended item to be recommended to a user of an item provided by an electronic means,
estimating a theme of interest, which is a theme of a recommended item to be presented to the user as a recommendation, based on a selection history of items by the user and a theme associated with the items;
for each of the estimated themes of interest, estimating the importance of each item in the theme;
determining a recommended item for the user for each of the themes of interest based on the ranking result of the importance;
Program for.

Images