JP7348117B2 - Information processing device, information processing method, and information processing program - Google Patents

Description

The present invention relates to an information processing apparatus, an information processing method, and an information processing program that determine the deterioration state of a selection model for selecting content to be recommended to a user.

BACKGROUND ART Conventionally, an information processing apparatus is known that recommends content such as news and advertisements to a terminal device owned by a user via the Internet (see, for example, Patent Document 1).
In such conventional information processing devices, a score of the content for the user is calculated using a user feature vector indicating the user's characteristics and a content feature vector indicating the content feature, and the content is arranged in descending order of the score. Instead, it is delivered (recommended) to the user's terminal device.

JP2010-262383A

In an information processing device that distributes recommendations as described above, it is important to check whether appropriate content is being recommended. In conventional information processing devices, for example, CTR (Click Through Rate) is monitored, and when the CTR decreases, it is determined that there is an abnormality in the recommendation status. However, depending on the content, the CTR varies greatly due to current events, etc., and it is difficult for conventional information processing devices to accurately determine whether the recommendation status is good.

An object of the present invention is to provide an information processing device, an information processing method, and an information processing program that can appropriately determine the deterioration state of a selection model for selecting content to be recommended to a user.

The information processing device of the present invention includes an acquisition unit that acquires a content viewing history by a user, the viewing history and content content information of each of a plurality of contents to be distributed, and an acquisition unit that acquires a content viewing history by a user; a first selection unit that selects the plurality of first contents to be distributed to the users using a first selection model that outputs a plurality of first contents; and a first selection unit configured by a different algorithm from the first selection model. a second selection unit that selects at least one second content to be distributed to the user using a two-selection model; and a user who owns recommendation information including the selected first content and second content. It includes a recommendation distribution unit that distributes to a terminal, and a determination unit that determines a deterioration state of the first selected model based on the viewing history regarding the first content and the second content.

In the present invention, a first selection model that selects content to be recommended to a user, and a second selection model configured by a different algorithm from the first selection model are used to select a plurality of first contents and at least one second selection model. The content is selected and distributed to the user terminal, and the deterioration state of the first selected model is determined based on the viewing history regarding the first content and the second content. Thereby, it is possible to appropriately determine the deterioration state of the selection model for selecting content to be recommended to the user while suppressing the influence of current events and the like.

1 is a schematic diagram showing an information processing system according to an embodiment of the present invention. FIG. 1 is a block diagram showing a schematic configuration of a server according to the present embodiment. 7 is a flowchart showing content recommendation processing in the method for determining the deterioration state of the first selected model in the present embodiment. The figure which shows an example of the recommendation display page of the content of this embodiment. FIG. 7 is another diagram showing an example of a content recommendation display page according to the present embodiment. 7 is a flowchart showing a process for determining the deterioration state of the first selected model in the present embodiment.

An embodiment according to the present invention will be described below.
[Overview of information processing system]
FIG. 1 is a schematic diagram showing an information processing system of this embodiment.
As shown in FIG. 1, the information processing system of this embodiment includes a server 10, which is an information processing device, and a plurality of terminal devices 20 connected to the server 10 via the Internet. This information processing system is a recommendation distribution system that recommends news content as content from the server 10 to the terminal device 20, and recommends content according to the characteristics of the user who owns the terminal device 20.
Hereinafter, such an information processing system, particularly the server 10, will be described in detail.

[Configuration of server 10]
FIG. 2 is a block diagram showing a schematic configuration of the server 10.
The server 10 is made up of a general computer, and as shown in FIG. 2, includes various parts that make up the computer, such as a communication section 11, a storage section 12, a processor 13, and the like. Note that the number of computers that constitute the server 10 is not particularly limited. In this embodiment, in order to simplify the explanation, an example is shown in which the server 10 is configured by one computer, but the server 10 may also be a cloud server configured by connecting a plurality of computers via a network.

The communication unit 11 is connected to the Internet and communicates with each device such as the terminal device 20 via the Internet.
The storage unit 12 records various information and information processing programs for controlling the server 10.
The storage unit 12 also includes databases such as a content storage unit 121 that records content to be distributed to the terminal device 20, a user information storage unit 122 that stores user information regarding users, and databases that are created for selecting content. A first selection model 123 and a second selection model 124 are stored.
Here, an example is shown in which the storage unit 12 of the server 10 is provided with a content storage unit 121, a user information storage unit 122, a first selection model 123, and a second selection model 124. The information and model may be recorded in another data server or cloud storage that is communicably connected.

The content storage unit 121 is a database in which content that can be distributed to the terminal device 20 is stored, and specifically, a plurality of pieces of content information are recorded. These content information may be generated by the administrator of the server 10, for example, or may be transmitted from an editor terminal owned by the content editor. Alternatively, the server 10 may crawl and collect predetermined web content on the Internet.
The content information includes content ID, submission date and time information, content content information, and the like.
Content ID is identification information that identifies content.
The submission date and time information is the date and time when the content was created or submitted.
The content content information is the content of the content to be distributed to the user, that is, the submitted article, and is composed of, for example, text data, image data, and the like.
In this embodiment, the content is not particularly limited, and may be, for example, an article such as news, an advertisement for a product or service, a recommended video on a video site, etc. good. Such content includes text data that includes multiple keywords.

A plurality of pieces of user information are recorded in the user information storage section 122. This user information includes a user ID, user attribute information, browsing history information, and the like.
The user ID is identification information that identifies the user.
User attribute information records various attributes of the user. For example, personal information such as the user's gender, age group, and location may be included, and preferences such as the user's hobbies may be recorded.
Browsing history information is history information of content that is selected (browsed) by the user from content recommended by the server 10 .
Note that the user information may also include a history of search keywords in search processing performed by the user by operating the terminal device 20, a history of buying and selling products via the Internet, a history of browsing web content by the user, etc. .

The first selection model 123 is a program generated by machine learning using AI such as deep learning based on teacher data, and uses viewing history information and content content information as input to create a first content that is recommended to the user. This is a model that makes selections and outputs the results. This first selection model 123 is a model generated by a known machine learning algorithm, and can be generated using, for example, a random forest.
The second selection model 124 is a program configured by a different algorithm from the first selection model 123, and selects second content to be recommended to the user in response to a predetermined input, and outputs the result.
Details of the first selection model 123, the second selection model 124, and the first content and second content will be described later.

The processor 13 is composed of an arithmetic circuit such as a CPU, and a recording circuit such as a RAM. The processor 13 loads the information processing program recorded in the storage unit 12 into the RAM, and executes various processes in cooperation with the program loaded into the RAM.
Then, by reading and executing the information processing program, the processor 13 functions as a user information acquisition unit 131, a first selection unit 132, a second selection unit 133, a distribution unit 134, and a determination unit 135, as shown in FIG. Function.

The user information acquisition unit 131 also functions as an acquisition unit of the present invention and acquires user information from the terminal device 20. That is, when the user selects and views any of the contents recommended from the server 10 to the terminal device 20, the user information acquisition unit 131 stores the selected content (selected content) and the selected content. Get the date and time. That is, the viewing history information of the user's content is acquired.
In addition, the user information acquisition unit 131 may acquire user attributes input by the user's input operation on the terminal device 20, or acquire other behavior histories such as search history.

The first selection unit 132 uses the above-described first selection model 123 to select a plurality of first contents to be recommended to the user.
The first selection unit 132 reads out the viewing history information of the user information from the user information acquisition unit 131 and the contents of each content stored in the content storage unit 121 and inputs them into the first selection model 123 .
Thereby, the first selection model 123 calculates a user feature vector based on the input browsing history information and a content feature vector based on the content content information, and selects a predetermined number of contents in descending order of the score indicating the user's interest level. Output.
Note that in this embodiment, the first selection unit 132 inputs browsing history information and content content information into the first selection model 123, but the first selection unit 132 inputs browsing history information and content content information into the first selection model 123. A feature vector or a content feature vector based on content content information may be calculated and input to the first selection model 123. In this case, the first selection model 123 receives the user feature vector and the content feature vector as input, and outputs a predetermined number of pieces of content information that are of high interest to the user.

Here, the first content selected by the first selection model 123 is content that is estimated to be of interest to the user. The selection of the first content is then influenced by the state of the first selection model 123. In other words, in a state where the first selection model 123 is not degraded, the content (first content) output in response to the input browsing history information is content that matches or substantially matches the interests of the actual user. Appropriate recommendations can be made that correspond to the interests of people (recommendation accuracy is high). However, if the first selection model 123 deteriorates, the above-mentioned recommendation accuracy will decrease, and there is a possibility that content that the user is interested in will not be selected.

The second selection unit 133 uses the second selection model 124 described above to select one second content to be recommended to the user. Here, as described above, the second selection model 124 is a program configured using a different algorithm from the first selection model 123, and for example, selects one second content using a simple algorithm such as the latest arrival order. .
The second content selected by the second selection unit 133 is content unrelated to the user's interests, and is content selected for determining the deterioration state of the first selection model 123, which will be described later.
Note that the selection of the second content is not affected by the state of the first selection model 123. In other words, even if the first selection model 123 deteriorates, the selection of the second content is always performed using a simple and predetermined algorithm, such as in order of newest content.

The distribution unit 134 determines the recommendation order of the contents selected by the first selection unit 132 and the second selection unit 133, selects the contents in the recommendation order, and recommends the contents to the terminal device 20.
The determination unit 135 determines the deterioration state of the first selected model 123 based on the viewing history regarding the first content and second content recommended to the terminal device 20 by the distribution unit 134.
Note that detailed processing by the distribution unit 134 and determination unit 135 will be described later.

[Configuration of terminal device 20]
The terminal device 20 is a computer managed by a user, and is configured by, for example, a smartphone, a tablet terminal, a personal computer, or the like.
Although illustration of the specific configuration of the terminal device 20 is omitted, the terminal device 20 has the basic configuration of a general computer. That is, the terminal device 20 includes an input operation unit that receives operations from an operator, a display that displays image information, a recording device that records various information, and an arithmetic circuit (such as a CPU) that processes various information.

[Information processing method]
Next, a method for determining the deterioration state of the first selection model 123 in the information processing system of this embodiment will be described, particularly an information processing method in the server 10.
Although the details will be described later, in this embodiment, in order to determine the deterioration state of the first selection model 123, recommendations are distributed for the first content and the second content, and the viewing history of the first content and the second content is get. At this time, it is not necessary to distribute the first content and the second content to all terminal devices 20. For example, the first content and the second content may be recommended and distributed to a certain percentage of the terminal devices 20, and the first content may be recommended and distributed to the other terminal devices 20 as usual. As described above, it is possible to determine the deterioration state of the first selection model 123 by distributing recommendations of the first content and the second content only to a certain percentage of the terminal devices 20. The fixed proportion of terminal devices 20 does not need to be fixed and may be changed randomly.

[Recommendation processing of first content and second content]
FIG. 3 is a flowchart showing content recommendation processing in this embodiment.
Here, in implementing the recommendation process, user information is acquired in advance and recorded in the user information storage section 122 of the storage section 12. That is, it is assumed that the history acquisition step of acquiring the browsing history has been performed in advance.

The server 10 receives request information requesting content recommendation and a user ID from the terminal device 20 (step S1).
Thereafter, the first selection unit 132 reads the user information corresponding to the user ID received together with the request information and each piece of content content information recorded in the content storage unit 121 and inputs them into the first selection model 123. Thereby, the first selection unit 132 selects the plurality of contents output from the first selection model 123 as the first contents (step S2).
Next, the second selection unit 133 selects one second content using the second selection model 124 (step S3).

Next, the distribution unit 134 determines the recommendation order of the first content selected in step S2 and the second content selected in step S3 (step S4).
As described above, a plurality of first contents are selected in step S2, and one second content is selected in step S3. The distribution unit 134 determines the recommendation order of the first content in descending order of the score calculated in step S4, and inserts the recommendation order of the second content into a part thereof, thereby determining the recommendation order of the first content and the second content. Determine.
In this embodiment, as an example, the second content is inserted between the first content with the 20th highest score and the first content with the 21st highest score. Here, the 21st item is not displayed in the initial state of the recommendation display field, which will be described later, but is displayed when the recommendation display field is scrolled downward.
Then, the distribution unit 134 selects the first content and the second content in the recommendation order determined in step S4, and transmits them to the terminal device 20 (step S5).

FIG. 4 is a diagram showing an example of a content recommendation display page 30 distributed to the terminal device 20 in this embodiment.

The recommendation display page 30 shown in FIG. 4 is a display example of a news content viewing application installed on a mobile terminal such as a smartphone. Typically, such content viewing applications display a "recommended" tab that recommends news content based on user information, a "TOP" tab that displays the latest news in order of arrival, tabs that correspond to various genres, etc. By selecting one of these, news content corresponding to the tab will be recommended.
Here, a description will be given of the recommendation display field 31 that is displayed when the "recommendation" tab that recommends news content based on user information is selected.

In this embodiment, as shown in FIG. 4, a plurality of item images IX (I1, I2) indicating the first content and second content recommended to the user are displayed in the recommendation display field 31. Link information corresponding to the content is associated with each item image IX, and by selecting (for example, tapping) an item image IX by a user operation, the display changes to the content corresponding to the item image IX.
The number of item images IX displayed at one time in the recommendation display field 31 is predetermined, and is eight in the example of FIG. When the user performs an operation of scrolling the recommendation display field 31 downward, item images IX arranged in the recommendation display field 31 are sequentially added. That is, although the number of item images IX displayed on the screen is eight, the number (total number) of item images IX placed in the recommendation display field 31 increases as they are sequentially added.

In other words, in the initial state where the recommendation display field 31 is not scrolled downward, as shown in FIG. 4, the item image I1 corresponding to the first content (C1 to C8), which is the top 8 in the recommendation order, is displayed. Ru. Then, by performing the scroll operation, the distribution unit 134 adds item images IX corresponding to a predetermined number of contents, but at this time, the item image I2 of the second content is placed 21st in the recommendation order. The first content and the second content are selected as follows.
FIG. 5 is a display example when the recommendation display field 31 is scrolled downward and the item image I2 of the second content is displayed. When the recommendation display field 31 is scrolled downward, as shown in FIG. An item image I2 corresponding to (C') is displayed.

Note that the arrangement position of the item image I2 corresponding to the second content is not particularly limited to the 21st position. The item image I2 corresponding to the second content may be placed at any position except for the topmost part in the initial state. Further, the arrangement position of the item image I2 corresponding to the second content does not need to be a fixed position, and may be changed randomly.

After that, when any of the contents arranged in the recommendation display field 31 is selected by the user's operation on the terminal device 20, selected content information (selected content) indicating the selected content (selected content) is displayed from the terminal device 20 For example, the content ID) is sent to the server 10.

The user information acquisition unit 131 of the server 10 determines whether selected content information has been received from the terminal device 20 (step S6).
If NO is determined in step S6, that is, if the selected content information is not received and the user has finished viewing the content on the terminal device 20, the process is terminated.
On the other hand, if the determination in step S6 is YES, that is, if the selected content information is received, the user information acquisition unit 131 updates the viewing history information (step S7) and returns to step S2.

[Determination process of deterioration state of first selection model 123]
FIG. 6 is a flowchart showing a process for determining the deterioration state of the first selected model 123 in this embodiment.
The user information acquisition unit 131 of the server 10 acquires viewing history information of the user's content (step S11). At this time, the user information acquisition unit 131 acquires both viewing history information regarding the first content and viewing history information regarding the second content.
Then, the determining unit 135 of the server 10 determines whether a predetermined time has elapsed (step S12). The predetermined time corresponds to a time interval at which the deterioration state of the first selected model 123 is determined. This time may be configurable or changeable by an administrator.
If NO is determined in step S12, that is, if the predetermined time has not elapsed, the process returns to step S11.

On the other hand, if it is determined as YES in step S12, that is, if the predetermined time has passed, the determination unit 135 calculates the amount of change N1 in the number of views of the first content (step S13), and The amount of change N2 in the number of times is calculated (step S14). Note that step S13 and step S14 may be executed in parallel or in reverse order (step S13 after step S14).

In steps S13 and S14, the change amount N1 regarding the first content and the change amount N2 regarding the second content may be calculated based on the number of views per unit period, or the first content relative to the total number of views per unit period. It may be calculated based on the number of views of the second content or the ratio of the number of views of the second content. The unit period at this time may be the same as or different from the above-described predetermined time.
In steps S13 and S14, the amount of change N1 and the amount of change N2 are, for example, amounts of change obtained by comparing the number of views or the ratio of the number of times of viewing in two consecutive unit periods.
Note that each time the number of views or the ratio of the number of views in a unit period is calculated, instead of targeting two consecutive unit periods, the number of views or the ratio of the number of views in a unit period is recorded in memory. The number of views or the percentage of the number of views may be read out. In this case, the number of views or the ratio of the number of views in the unit period newly calculated in the determination process is compared with the number of views or the ratio of the number of views in the past unit period read from the memory, and the amount of change N1 and the ratio of the number of views are determined. The quantity N2 can be determined.
The amount of change N1 and the amount of change N2 calculated in this way include the direction of change (increase or decrease), the width of change, etc.

Next, the determination unit 135 of the server 10 determines whether the first selection model 123 has deteriorated based on the amount of change N1 regarding the first content calculated in step S13 and the amount of change N2 regarding the second content calculated in step S14. It is determined whether or not (step S15).
In step S15, the determination unit 135 determines that when the amount of change N1 regarding the first content indicates a decrease in the number of views of the first content, and the amount of change N2 regarding the second content does not indicate a decrease in the number of times the second content has been viewed. In this case, it is determined that "the first selected model 123 has deteriorated".

On the other hand, in step S15, if the above-mentioned condition is not satisfied, such as when the amount of change N1 regarding the first content does not indicate a decrease in the number of views of the first content, the determination unit 135 determines that "the first selection model 123 is It is determined that there is no deterioration. For example, if the amount of change N1 related to the first content indicates a decrease in the number of views of the first content, and the amount of change N2 related to the second content also indicates a decrease in the number of times the second content is viewed, the first selection model 123 It is determined that "the first selected model 123 has not deteriorated" because this is not an influence due to the state of deterioration of the model 123, but an overall influence due to current events, etc.
That is, in step S15, the determination unit 135 detects a relative change by comparing the amount of change N1 regarding the first content with the amount of change N2 regarding the second content, and detects whether the first selected model 123 has deteriorated. Determine whether or not there is.

If the determination in step S15 is NO, that is, if it is determined that the first selected model 123 has not deteriorated, the process is terminated.
On the other hand, if it is determined as YES in step S15, that is, if it is determined that the first selected model 123 has deteriorated, the first selection unit 132 recreates the first selected model 123 (step S16 ), the process ends.
Note that the first selection model 123 may be recreated in any manner. For example, the first selection model 123 may be recreated by performing machine learning again. Alternatively, an alternative selection model may be stored in the storage unit 12, and the first selection model 123 may be changed to the alternative selection model.

[Actions and effects of this embodiment]
The server 10 of this embodiment includes a storage unit 12 and a processor 13, and the processor 13 reads the information processing program stored in the storage unit 12, and the user information acquisition unit 131 (acquisition unit) and the first It functions as a selection unit 132, a second selection unit 133, a distribution unit 134 (recommendation distribution unit), and a determination unit 135. The user information acquisition unit 131 acquires the viewing history of content by the user. The first selection unit 132 uses a first selection model 123 that inputs the viewing history and the content content information of each of the plurality of contents to be distributed, and outputs the plurality of first contents according to the viewing history. , select the plurality of first contents to be distributed to the user. The second selection unit 133 uses a second selection model 124 configured by a different algorithm from the first selection model 123 to select at least one second content to be distributed to the user. The distribution unit 134 distributes recommendation information including the selected first content and second content to a user terminal owned by the user. The determining unit 135 determines the deterioration state of the first selected model 123 based on the viewing history regarding the first content and the second content.
In other words, in this embodiment, recommendations are distributed for first content that is presumed to be of interest to the user and second content that is unrelated to the user's interests, and based on the browsing history of the first content and the second content. Then, the state of deterioration of the first selected model 123 is determined. Thereby, it is possible to accurately determine whether the recommendation by the first selection model 123 is in good condition.

In the present embodiment, the determination unit 135 calculates the number of views per unit period based on the viewing history per unit period regarding the first content and the second content, and calculates the number of views per unit period based on the number of views regarding the first content and the second content. Then, the state of deterioration of the first selected model 123 is determined. Therefore, a relative change in the viewing history regarding the first content and the second content can be detected.

In this embodiment, the determination unit 135 determines that the first selection model 123 has deteriorated when the number of views related to the first content decreases and the number of views related to the second content does not decrease. Thereby, it can be determined whether the viewing statuses of the first content and the second content are decreasing in conjunction with each other.

[Modified example]
Note that the present invention is not limited to the embodiments described above, but includes the following modifications as long as the object of the present invention can be achieved.

[Modification 1]
In the above embodiment, an example is shown in which the number of views of the first content and the second content per unit period is calculated in steps S13 and S14 based on the viewing history information per unit period acquired by the user information acquisition unit 131. However, for example, the user information acquisition unit 131 may acquire the number of views of the first content and the second content per unit period as the viewing history information. Thereby, the processing load on the determination unit 135 can be reduced.

[Modification 2]
In the above embodiment, in step S15, the deterioration state of the first selected model 123 is determined based on the amount of change N1 regarding the first content calculated in step S13 and the amount of change N2 regarding the second content calculated in step S14. showed that. In other words, although an example has been shown in which the determination unit 135 constantly calculates and monitors the change amount N1 regarding the first content and the change amount N2 regarding the second content, the determination unit 135 always calculates only the change amount N1 regarding the first content, and monitors the change amount N1 regarding the first content. The state of the first selection model 123 is monitored based on the amount of change N1 related to the content, and only when the amount of change N1 related to the first content is decreased and an abnormality is considered to have occurred, the amount of change N2 related to the second content is calculated. , the deterioration state of the first selected model 123 may be determined.

[Modification 3]
In the embodiment described above, an example has been shown in which when it is determined in step S16 that the first selected model 123 has deteriorated, the first selected model 123 is re-created in step S17, but the present invention is not limited to this.
For example, if it is determined in step S16 that the first selected model 123 has deteriorated, the administrator may be notified of the abnormality using text information or audio information, or the first selected model 123 may be The selection model may be changed to a selection model configured by a general selection algorithm such as order of popularity or order of popularity.

[Modification 4]
In the above embodiment, an example has been shown in which the first selection model 123 and the second selection model 124 are configured by a different algorithm from the first selection model 123, but the first selection model 123 and the second selection model 124 It may also include a third selection model that is configured by a different algorithm and that selects the third content. In this case, for example, by distributing recommendations for the first content, second content, and third content, and evaluating browsing history information regarding the first content, second content, and third content, the first selection can be made more accurately. It is expected that the deterioration state of the model 123 can be determined.

DESCRIPTION OF SYMBOLS 10... Server (information processing device), 12... Storage unit, 13... Processor, 20... Terminal device, 30... Recommendation display page, 31... Recommendation display field, 121... Content storage unit, 122... User information storage unit, 123... First selection model, 124... Second selection model, 131... User information acquisition unit (acquisition unit), 132... First selection unit, 133... Second selection unit, 134... Distribution unit, 135... Judgment unit, IX (I1 , I2)...Item image.

Claims (6)

an acquisition unit that acquires the content viewing history by the user;
Deliver to the user using a first selection model in which the viewing history and content information of each of the plurality of contents to be distributed are input, and the plurality of first contents according to the viewing history are output. a first selection unit that selects a plurality of first contents;
a second selection unit that selects at least one second content to be distributed to the user using a second selection model configured by a different algorithm from the first selection model;
a recommendation distribution unit that distributes recommendation information including the selected first content and second content to a user terminal owned by the user;
and a determination unit that determines a deterioration state of the first selected model based on the number of times the first content is viewed by the user per unit period and the number of times the second content is viewed per unit period by the user. information processing equipment. The information processing device according to claim 1,
The information processing device is characterized in that the acquisition unit acquires the number of times of viewing per unit period as the viewing history. The information processing device according to claim 1,
The information processing device is characterized in that the determination unit calculates the number of times of viewing per unit period based on the viewing history per unit period regarding the first content and the second content. The information processing device according to claim 1,
The determining unit determines that the first selected model is degraded when the number of views regarding the first content decreases and the number of views regarding the second content does not decrease. information processing equipment. An information processing method for distributing recommendation information to a user terminal by a computer, the method comprising:
The computer includes an acquisition unit, a first selection unit, a second selection unit, a recommendation distribution unit, and a determination unit,
The computer includes:
a step in which the acquisition unit acquires a content viewing history by the user;
The first selection unit inputs the viewing history and content content information of each of the plurality of contents to be distributed, and selects a first selection model in which the plurality of first contents according to the viewing history is output. selecting a plurality of the first contents to be used and distributed to the user;
the second selection unit selecting at least one second content to be distributed to the user using a second selection model configured by a different algorithm from the first selection model;
a step in which the recommendation distribution unit distributes recommendation information including the selected first content and second content to a user terminal owned by the user;
the determination unit determines a deterioration state of the first selected model based on the number of times the first content is viewed per unit period by the user and the number of times the second content is viewed per unit period . An information processing method characterized by carrying out. An information processing program that is read and executed by a computer,
An information processing program that causes the computer to function as the information processing device according to any one of claims 1 to 4.

Images