KR102619044B1 - Recommending method based on machine-learning and system thereof - Google Patents

Abstract

A machine learning-based recommendation method and system are provided. A recommendation method according to some embodiments of the present disclosure includes obtaining a sequence-to-sequence-based deep learning model learned by performing a next consumption content prediction task using the user's content consumption history data, It may include obtaining information about the content sequence consumed by the user and determining recommended content for the user by predicting the next consumed content from the content sequence through a learned deep learning model. According to this method, customized content preferred by the user can be accurately recommended.

Description

Machine learning-based recommendation method and system {RECOMMENDING METHOD BASED ON MACHINE-LEARNING AND SYSTEM THEREOF}

This disclosure relates to a method and system for recommending items (e.g., content) from various domains to users using machine learning techniques.

As OTT (Over-The-Top) services become more popular, interest in technology that recommends customized content to users is rapidly increasing. In addition, this recommendation technology is widely used not only in OTT services but also in the field of recommending products, music, news, etc.

Meanwhile, most of the recommendation techniques used in the OTT service field are based on analysis of the user's content consumption history. For example, the existing recommendation technique analyzes the user's content consumption history to identify the characteristics of the content (e.g., genre, actor, topic, etc.) mainly consumed by the user and recommends content by reflecting this (e.g., Recommend content with similar characteristics).

However, the illustrated recommendation technique does not take into account the sequential consumption pattern inherent in the user's content consumption history, so it has a clear limitation in that the recommendation accuracy is not high. In addition, the illustrated recommendation technique is not free from the cold start problem, which is one of the chronic problems in the recommendation field.

Korean Patent Publication No. 10-2022-0080563 (published on June 14, 2022)

The technical problem to be solved through several embodiments of the present disclosure is to provide a method for accurately recommending customized content preferred by a user and a system for performing the method.

Another technical problem to be solved through some embodiments of the present disclosure is to provide a recommendation method that is robust to the cold start problem and a system for performing the method.

Another technical problem to be solved through some embodiments of the present disclosure is to provide a method for accurately performing multi-domain recommendation using history data of different domains and a system for performing the method.

The technical problems of the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the description below.

The machine learning-based recommendation method according to some embodiments of the present disclosure for solving the above-described technical problem is a method performed by at least one computing device, and is a method that uses a sequence-to-recommendation method learned using the user's content consumption history data. - Obtaining a sequence-based deep learning model - The deep learning model is learned through a next consumption content prediction task - Obtaining information about the content sequence consumed by the user and the deep learning and determining recommended content for the user by predicting next consumed content from the content sequence through a model.

In some embodiments, the deep learning model may be implemented based only on the decoder among the encoder and decoder of a transformer.

In some embodiments, the step of acquiring the deep learning model includes extracting a content sequence sample from the content consumption history data, and selecting the content sequence sample based on a determination that the length of the content sequence sample is greater than or equal to a reference value. It may include generating a training set by dividing it into subsequence samples with a length less than a reference value, and learning the deep learning model using the training set.

In some embodiments, acquiring the deep learning model includes extracting a content sequence sample from the content consumption history data, removing content whose popularity is higher than a threshold value from the content sequence sample to create a training set, and It may include training the deep learning model using the training set.

In some embodiments, determining the recommended content may include determining the recommended content by removing content whose popularity is greater than or equal to a threshold value from the content sequence and predicting the next consumed content using the remaining content sequence. You can.

In some embodiments, the deep learning model includes a prediction layer configured to output a confidence score for each content, and the step of determining the recommended content includes confidence scores for a plurality of contents pre-registered through the prediction layer. It may include calculating and determining a content whose confidence score is equal to or higher than a standard value among the plurality of contents as the recommended content.

In some embodiments, determining the recommended content includes determining the recommended content through the deep learning model that learned the content consumption history data when the amount of content consumption history data is greater than a reference value. And, if the amount of content consumption history data is less than the standard value, the recommended content may be determined through a reinforcement learning model that learned the user's feedback.

A machine learning-based recommendation method according to some other embodiments of the present disclosure for solving the above-described technical problem is a method performed by at least one computing device, and includes a sequence learned using the user's item consumption history data- Obtaining a two-sequence-based deep learning model - the deep learning model is learned through a next consumption item prediction task -, obtaining information about the sequence of items consumed by the user, and the deep learning model It may include determining a recommended item for the user by predicting the next consumption item from the item sequence through a learning model.

In some embodiments, the item consumption history data includes item consumption history data of a first domain and item consumption history data of a second domain, and the step of acquiring the deep learning model includes a plurality of items in the item consumption history data. generating a training set by extracting item sequence samples - at least some of the item sequence samples include items of the first domain and items of the second domain - and performing the deep learning using the training set. It may include the step of training the model.

In some embodiments, the item consumption history data includes item consumption history data of a first domain and item consumption history data of a second domain, the item sequence includes items of the first domain, and the recommended item The step of determining may include determining the recommended item by predicting the next consumption item of the second domain through the deep learning model.

A recommendation system according to some embodiments of the present disclosure for solving the above-described technical problem includes one or more processors and a memory for storing instructions, wherein the one or more processors execute the stored instructions to create user content. An operation of acquiring a sequence-to-sequence-based deep learning model learned using consumption history data - the deep learning model is learned through a next consumption content prediction task -, content sequence consumed by the user An operation of obtaining information about and determining recommended content for the user can be performed by predicting the next consumption content from the content sequence through the deep learning model.

A recommendation system according to some other embodiments of the present disclosure for solving the above-described technical problem includes one or more processors and a memory for storing instructions, wherein the one or more processors execute the stored instructions to provide a user's information. An operation of acquiring a sequence-to-sequence-based deep learning model learned using item consumption history data - the deep learning model is learned through a next consumption item prediction task -, items consumed by the user It may include an operation of obtaining information about the sequence and an operation of determining a recommended item for the user by predicting the next consumption item from the item sequence through the deep learning model.

A computer program according to some embodiments of the present disclosure for solving the above-mentioned technical problems is combined with a computing device to obtain a sequence-to-sequence-based deep learning model learned using the user's content consumption history data. - the deep learning model is learned through a next consumption content prediction task -, obtaining information about the content sequence consumed by the user and the next consumption from the content sequence through the deep learning model It may be stored in a computer-readable recording medium to execute the step of determining recommended content for the user by predicting content.

A computer program according to several other embodiments of the present disclosure for solving the above-mentioned technical problems is combined with a computing device to create a sequence-to-sequence-based deep learning model learned using the user's item consumption history data. Obtaining - the deep learning model is learned through a next consumption item prediction task -, acquiring information about the sequence of items consumed by the user, and selecting the next item from the sequence through the deep learning model. It may be stored in a computer-readable recording medium to execute the step of determining recommended items for the user by predicting consumption items.

According to some embodiments of the present disclosure, a deep learning model (i.e., sequence-to-sequence based deep learning model) is developed by performing a next consumption content prediction task using content sequence samples extracted from the user's content consumption history data. learning model) can be learned. In this case, content recommendation accuracy can be greatly improved because the deep learning model can accurately predict the next consumed content by capturing the sequential consumption pattern inherent in the user's content consumption history.

Additionally, a training set can be created through a cleaning process that divides content sequence samples whose length is greater than the reference value into sub-sequence samples. In this case, since distortion (misrecognition) learning due to noise included in long sequence samples can be prevented in advance, the recommendation accuracy of the deep learning model can be further improved.

Additionally, a training set can be created through congestion processing that removes content whose popularity is above a threshold value from content sequence samples. In this case, the recommendation accuracy of the deep learning model can be further improved because distortion (misconception) learning caused by mega content that reflects users' universal tastes can be prevented in advance.

Additionally, content embedding can be created based on various information about the content (e.g., identification code, title, representative image, etc.). In this case, the deep learning model is trained to predict the next consumption content by comprehensively considering various information about the contents, so the recommendation accuracy of the deep learning model can be further improved.

Additionally, if the amount of data on the user's content consumption history is insufficient, content recommendation may be performed using a reinforcement learning model, and in the opposite case, content recommendation may be performed using a deep learning model. In other words, in situations where the performance of the deep learning model is predicted to decline, the user's satisfaction with the recommended content can be maintained at a consistently high level by recommending content using a reinforcement learning model.

In addition, a deep learning model is learned using consumption history data of multi-domains, and a multi-domain recommendation service can be provided through the learned deep learning model. For example, a service that recommends delivery food or products to a user consuming content may be provided. In this case, a new (fresh) experience with the recommended service may be provided to the user. In addition, because the consumption history data required for model learning can be prepared quickly, the cold start problem, one of the chronic problems in the recommendation field, can be easily solved.

The effects according to the technical idea of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below.

1 is an exemplary configuration diagram showing the operating environment of a recommendation system according to some embodiments of the present disclosure.
FIG. 2 is an exemplary diagram schematically showing a process in which a recommendation system recommends content based on a reinforcement learning model according to some embodiments of the present disclosure.
FIG. 3 is an exemplary diagram schematically showing a process in which a recommendation system recommends content based on a deep learning model according to some embodiments of the present disclosure.
4 and 5 are exemplary diagrams for schematically explaining a process in which a recommendation system according to some embodiments of the present disclosure provides a multi-domain recommendation service.
6 to 8 are exemplary diagrams for explaining examples of a recommended content provision method and user feedback information according to some embodiments of the present disclosure.
9 is an example flowchart schematically illustrating a machine learning-based recommendation method according to some embodiments of the present disclosure.
10 to 12 are exemplary diagrams for explaining the operation and implementation method of a deep learning model according to some embodiments of the present disclosure.
Figure 13 is an example diagram for explaining the structure and learning method of a deep learning model according to some embodiments of the present disclosure.
FIG. 14 is an exemplary diagram for explaining a content embedding method according to some embodiments of the present disclosure.
FIG. 15 is an exemplary diagram for explaining a data purification method according to some embodiments of the present disclosure.
FIG. 16 is an exemplary diagram for explaining a data purification method according to some other embodiments of the present disclosure.
FIG. 17 is an exemplary diagram illustrating a problem caused by new content registration and a method for solving the problem according to some embodiments of the present disclosure.
FIG. 18 is an exemplary diagram for explaining a method of operating a recommendation model according to some embodiments of the present disclosure.
FIG. 19 is an exemplary diagram schematically illustrating a machine learning-based multi-domain recommendation method according to some embodiments of the present disclosure.
FIG. 20 is an exemplary diagram illustrating an item embedding method in a multi-domain environment according to some embodiments of the present disclosure.
FIG. 21 is an exemplary diagram illustrating an item embedding method in a multi-domain environment according to some other embodiments of the present disclosure.
FIG. 22 is an exemplary diagram illustrating the structure of a deep learning model and a method for determining a recommended item in a multi-domain environment according to some embodiments of the present disclosure.
FIG. 23 is an exemplary diagram illustrating the structure of a deep learning model and a method for determining a recommended item in a multi-domain environment according to some other embodiments of the present disclosure.
24 illustrates an example computing device that can implement a recommendation system according to some embodiments of the present disclosure.

Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings. The advantages and features of the present disclosure and methods for achieving them will become clear by referring to the embodiments described in detail below along with the accompanying drawings. However, the technical idea of the present disclosure is not limited to the following embodiments and may be implemented in various different forms. The following examples are merely intended to complete the technical idea of the present disclosure and to cover the technical field to which the present disclosure belongs. is provided to fully inform those skilled in the art of the scope of the present disclosure, and the technical idea of the present disclosure is only defined by the scope of the claims.

In describing various embodiments of the present disclosure, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present disclosure, the detailed description will be omitted.

Unless otherwise defined, terms (including technical and scientific terms) used in the following embodiments may be used in a meaning that can be commonly understood by those skilled in the art in the technical field to which this disclosure pertains. It may vary depending on the intentions or precedents of engineers working in related fields, the emergence of new technologies, etc. The terminology used in this disclosure is for describing embodiments and is not intended to limit the scope of this disclosure.

The singular expressions used in the following embodiments include plural concepts, unless the context clearly specifies singularity. Additionally, plural expressions include singular concepts, unless the context clearly specifies plurality.

In addition, terms such as first, second, A, B, (a), (b) used in the following embodiments are only used to distinguish one component from another component, and the terms The nature, sequence, or order of the relevant components are not limited.

According to various embodiments of the present disclosure, a recommendation method and system based on a machine learning model may be provided. At this time, recommended items include items from various domains (or types), such as content (e.g., movies, dramas, TV programs, webtoons, music, news, advertisements, etc.), products, food (e.g., delivery food), etc. It can be included without limitation. However, in the following, for convenience of understanding, explanation will be made assuming that the recommended item is 'content'.

Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the attached drawings.

1 is an exemplary configuration diagram showing the operating environment of the recommendation system 10 according to some embodiments of the present disclosure.

As shown in FIG. 1, the recommendation system 10 may operate in an environment that provides content services to a plurality of user terminals 12-1 to 12-n. For example, the recommendation system 10 may operate in conjunction with, or be included in, a system 11 that provides an OTT (Over-The-Top) service or a content streaming service. However, the scope of the present disclosure is not limited thereto. Hereinafter, the reference number '12' will be used to refer to any user terminal (e.g., 12-1) or to refer collectively to all user terminals (12-1 to 12-n).

To provide convenience of understanding, the content service system 11 and the user terminal 12 will be explained first before the recommendation system 10.

The content service system 11 may be a computing device/system that provides content services to users. For example, the content service system 11 may be a system (platform) that streams requested content from among pre-registered content to the user terminal 12. In this case only, the content service system 11 may be named 'content streaming system (platform)', 'content streaming server', etc. The content service system 11 may be implemented in any form.

Content that is serviced (provided) may have various forms, such as video (image), image, audio (voice), text, multimedia, etc., and may take any form. Additionally, the serviced content may encompass various types of content, such as movies, dramas, TV programs, webtoons, music, news, advertisements, etc. However, in the following, in order to provide convenience of understanding, the explanation will be continued assuming that the serviced content is content in a playable form such as multimedia, video, music, etc.

Next, the user terminal 12 may refer to a terminal used by a user to receive a content service (or a service of another domain). For example, various types of computing devices such as smartphones, laptops, desktops, TVs, kiosks, digital signage, etc. may be included in the category of user terminal 12 without limitation.

The user can receive various contents through the terminal 12. For example, the user can access the content service page through the terminal 12, log in, consume desired content, and be provided with recommended content. Additionally, the user may provide various feedback on recommended content through the terminal 12.

For reference, the term 'consumption' can be interpreted to encompass meanings such as 'watching', 'listening', and 'appreciation'. For example, consuming video content may mean watching the content, and consuming audio content may mean listening to or appreciating the content. Additionally, the term 'consumption' refers to various types of actions (e.g., purchase, order, etc.) on specific items (e.g., content, products, food) that cause consumption of the user's resources (e.g., time, money, etc.). It can be interpreted as inclusive.

Next, the recommendation system 10 may be a computing device/system that recommends content (or items from another domain) to a user. For example, the recommendation system 10 may determine recommended content for the user and provide the recommended content (e.g., recommended content information) to the user directly or through the content service system 11. For example, recommended content may be provided (exposed) to the user by being displayed in the hero area of the content service page (e.g., the main service page provided through the content service app), as described in FIGS. 6 to 8. Please refer to the contents.

For reference, Figure 1 shows the content service system 11 and the recommendation system 10 as being implemented in separate physical computing devices/systems, but in some cases, the two systems 10 and 11 may be implemented in the same physical computing device/system. They may also be implemented in an integrated form within a device/system. For example, the recommendation system 10 may be implemented in the form of a module within the content service system 11.

Additionally, as described above, Figure 1 assumes that the recommended item is 'content'. Accordingly, if the recommended item is an item from another domain (type), the recommendation system 10 can perform recommendation in conjunction with a system providing the item. For example, if the recommended item is a 'product', the recommendation system 10 may perform product recommendation in conjunction with a system that provides a shopping service (or product purchase service). Similarly, if the recommended item is 'delivery food', the recommendation system 10 may perform food recommendation in conjunction with a system that provides a delivery service (or food ordering service).

The recommendation system 10 may determine recommended content (e.g., customized content) for the user using a machine-learning model. However, the specific method may vary depending on the embodiment.

In some embodiments, as shown in FIG. 2, the recommendation system 10 may determine recommended content through the reinforcement learning model 20. For example, the recommendation system 10 determines recommended content through the reinforcement learning model 20, updates (learns) the reinforcement learning model 20 using the user's feedback information, and creates the updated model 20. The process of determining recommended content can be repeatedly performed. As this process is repeated, the reinforcement learning model 20 can more accurately recommend user-customized content.

Feedback (e.g., user actions related to recommended content) information may include information about all types of feedback that can be obtained from users, such as direct (explicit) feedback, indirect (implicit) feedback, positive feedback, negative feedback, etc. You can. Examples of such feedback information include information related to content selection (e.g., whether a content page link was clicked, number of clicks, number of other content browsed/played/clicked before selection, time taken to select, etc.), information related to content playback/consumption (e.g., e.g., whether or not the play button was clicked, number of pauses during playback, continuous playback time, total playback time, whether played to the end, time taken to play, number of other contents browsed/played/clicked before playback, continued viewing when changing devices (whether or not the user clicked the save button, time taken to watch to the end, etc.), information related to saving (e.g., whether the save button was clicked or the unsave button was clicked, etc.), sharing related information, review-related information (e.g., whether a review was written, review content) etc.) may be included. However, the scope of the present disclosure is not limited thereto. Regarding feedback information, further refer to the descriptions of FIGS. 6 to 8.

The reward (or reward score) used for updating (learning) of the reinforcement learning model 20 can be calculated based on the example feedback information. However, the specific calculation method can be designed in various ways.

For example, the shorter the time it takes to select or play recommended content, the higher the reward value may be calculated.

As another example, the lower the number of contents selected or played by the user before selecting or playing the recommended content, the higher the reward value may be calculated.

As another example, the longer the playback time or continuous playback time of recommended content, the higher the reward value may be calculated.

As another example, the fewer the number of pauses, the higher the reward may be calculated.

As another example, the shorter the time it takes to watch recommended content to the end, the higher the reward value may be calculated.

As another example, when a user continues to view recommended content through another terminal, additional rewards may be granted.

As another example, if you play or add recommended content, a higher reward may be given than if you simply select it.

As another example, a reward for feedback information may be calculated based on various combinations of the above-described examples.

For reference, the recommendation system 10 may obtain the user's feedback information through a content service system 11 (e.g., a server providing a content service page, etc.) or directly from the user terminal 12. Any method may be used to obtain feedback information.

The reinforcement learning model 20 may be, for example, a model that operates based on the MAB (Multi-Armed Bandits) algorithm. For a more specific example, the reinforcement learning model 20 may be a model that operates based on Thompson sampling, UCB (Upper Confidence Bound), ε-greedy algorithm, etc. However, the scope of the present disclosure is not limited thereto. Anyone working in the relevant technical field will already be familiar with the operating principles and methods of the illustrated algorithms/models, so descriptions thereof will be omitted.

In some other embodiments, as shown in FIG. 3, the recommendation system 10 may determine recommended content through the deep learning model 30. For example, the recommendation system 10 can learn the deep learning model 30 using the user's content consumption history data 31. Additionally, the recommendation system 10 can determine recommended content by predicting the next consumption content from the user's recently consumed content using the learned deep learning model 30. The deep learning model 30 can accurately predict the next consumption content by learning the user's sequential consumption pattern embedded in the content consumption history data 31, which will be detailed later with reference to the drawings in FIG. 9 and below. Please explain clearly.

In some other embodiments, recommended content may be determined based on various combinations of the above-described embodiments. For example, the recommendation system 10 determines the first recommended content through the reinforcement learning model 20, determines the second recommended content through the deep learning model 30, and selects the first recommended content and the second recommended content. It can also be provided to users. At this time, the number ratio of the first recommended content and the second recommended content may be determined based on the performance of the reinforcement learning model 20 and/or the deep learning model 30. As another example, the recommendation system 10 determines first recommendation candidate contents through the reinforcement learning model 20, determines second recommendation candidate contents through the deep learning model 30, and first recommendation candidate contents. and content selected from among the second recommendation candidate content (e.g., overlapping content, etc.) may be determined as recommended content. As another example, the recommendation system 10 determines recommended content through the reinforcement learning model 20 under the first condition (situation) and determines the recommended content through the deep learning model 30 under the second condition (situation). It may be possible. For further information, please refer to the description of FIG. 18.

Meanwhile, in some embodiments, the recommendation system 10 may provide a multi-(cross) domain recommendation service to the user. For example, the recommendation system 10 can collect user history data generated in different domains and learn from it to provide a multi-domain recommendation service. In this case, because the data required for model learning can be quickly prepared, the cold start problem, one of the chronic problems in the recommendation field, can be easily solved. In order to provide easier understanding, the present embodiments will be further described with reference to FIGS. 4 and 5.

As shown in FIG. 4, the recommendation system 10 can collect user history data 41 to 43 generated in multiple domains (see D1 to D3). For example, the recommendation system 10 may collect various historical data 41 to 43 about the user through service systems (e.g., 11, etc.) of each domain. However, the scope of the present disclosure is not limited to this, and any method may be used to collect domain-specific history data.

Domains may be classified based on, for example, the type of recommended item, the type of service (or app) related to the item, etc., but the scope of the present disclosure is not limited thereto. Figure 4 assumes a multi-domain environment consisting of a content (or content service)-related domain, a product (or shopping service)-related domain, and a food (or delivery service)-related domain.

Next, the recommendation system 10 can train a deep learning model (e.g., 30) using the collected historical data (41 to 43) and provide a multi-domain recommendation service to the user through the learned deep learning model. there is. For example, as shown in FIG. 5, when a user is consuming content through a first terminal (12-1, e.g., TV), the recommendation system 10 recommends the user's second terminal (12-2, e.g., TV). , a deep learning model learns the behavior patterns of users who recommend delivery food through apps (51, 52) of other domains installed on smartphones (see 53, e.g., ordering delivery food when consuming video content). Case) Products related to the content being consumed can be recommended (see 54, e.g., when a deep learning model learns the behavior patterns of users who purchase products related to the content being consumed). In this case, a new (fresh) experience with the recommendation service can be provided to the user. The recommendation system 10 may provide such a recommendation service by linking with the service system (e.g., 11) of each domain or directly linking with the apps 51 and 52. However, the scope of the present disclosure is not limited thereto.

The method of performing multi-domain recommendation through a deep learning model will be described in more detail later with reference to FIGS. 19 to 23.

The above-described recommendation system 10 may be implemented with at least one computing device. For example, all functionality of recommender system 10 may be implemented in a single computing device, with a first functionality of recommender system 10 implemented in a first computing device and a second functionality implemented in a second computing device. It could be. Alternatively, specific functionality of recommender system 10 may be implemented on multiple computing devices.

Computing devices may include any device equipped with a computing function, and see FIG. 24 for an example of such a device.

For reference, a computing device is a collection of interacting various components (e.g., memory, processor, etc.), so in some cases, it may be called a 'computing system'. Of course, the term computing system can also encompass the concept of a collection of interacting computing devices.

As shown in FIG. 1, the content service system 11 and the user terminal 12, the recommendation system 10 and the user terminal 12, etc. can communicate through a network. Here, the network is implemented as all types of wired/wireless networks such as Local Area Network (LAN), Wide Area Network (WAN), mobile radio communication network, Wibro (Wireless Broadband Internet), etc. It can be.

So far, an exemplary operating environment of the recommendation system 10 according to some embodiments of the present disclosure has been schematically described with reference to FIGS. 1 to 5 . Hereinafter, with reference to FIGS. 6 to 8 , examples of recommended content provision (exposure) methods and user feedback information according to some embodiments of the present disclosure will be briefly described.

Hereinafter, for clarity of the present disclosure, the reference numbers of the models 20 and 30 may be omitted when not directly referring to the drawings, and depending on the embodiment, the reference numbers of the models 20 and 30 may change.

FIG. 6 illustrates a case where recommended content is provided through a profile selection page 60 among content service pages according to some embodiments of the present disclosure. The profile selection page 60 may be, for example, a page (screen) displayed on the user terminal 12 upon successful login (e.g., when the content service supports multi-profile), but the scope of the present disclosure is not this. It is not limited.

As illustrated in FIG. 6 , recommended content determined by the recommendation system 10 may be displayed in the hero area 64 of the profile selection page 60 . For example, information such as still images, short videos (clips), and text of recommended content may be displayed in the hero area 64. However, the scope of the present disclosure is not limited thereto.

For reference, the hero area 64 may refer to an area that occupies a relatively large proportion of the page 60 or an area that is visually prominent. Anyone working in the relevant technical field will already be familiar with the concept of the hero area, so explanation thereof will be omitted.

In these embodiments, the recommendation system 10 uses the user's login information to select a machine learning model (i.e., a customized/personalized machine learning model) corresponding to the user (or type of user), and selects the selected machine. Recommended content can be determined using a learning model. Alternatively, the recommendation system 10 may perform customized recommendation by inputting (feeding) (i.e., explicitly providing) the user's type information into the machine learning model.

Multiple profiles 61 to 66 may or may not correspond 1:1 to the user. However, hereinafter, for convenience of understanding, it is assumed that one profile (e.g., 61) refers to one user.

FIG. 7 illustrates a case where recommended content is displayed on the main service page 70 among content service pages according to some embodiments of the present disclosure. The main service page 70 may be, for example, a page (screen) displayed on the user terminal 12 when a profile (e.g., 61) is selected or login is successful, but the scope of the present disclosure is not limited thereto.

As shown in FIG. 7, the main service page 70 may include a category menu area 71, a hero area 72, and other content areas (e.g., 73). In addition, the main service page 70 may further include a representative information area 75 of recommended content, a play button 76, and a content indicator 77. The main service page 70 may provide a content search function according to categories, topics, etc., and may further provide a content search function (see 75) and a profile change function (see 74).

Similar to what was described above, recommended content may be displayed in the hero area 72 of the main service page 70. Additionally, representative information (e.g., title, genre, actors, etc.) of the recommended content 75 may be displayed near the hero area 72.

For reference, the content indicator 77 can be understood as having a search function for recommended content and the purpose of notifying the total number of recommended content (e.g., 6).

In this page 70, feedback information related to content playback may be generated, for example, when the user selects the play button 76.

In addition, feedback information related to content selection may be generated, for example, when the user selects (clicks) the representative information area 75 or the hero area 72 of the recommended content (e.g., recommended content in the hero area 72). (if the link to the page is embedded). In such a case, the screen of the user terminal 12 moves from the main service page 70 to the recommended content page (see 80 in FIG. 8), and feedback information related to content selection may also be generated.

8 illustrates a content page 80 according to some embodiments of the present disclosure. For example, the content page 80 may be a page (screen) displayed on the user terminal 12 when the user selects the corresponding content (e.g., selecting hero area 72, etc.), but the scope of the present disclosure is limited to this. It doesn't work.

As shown in FIG. 8, the content page 80 may include a category menu area 71, a hero area 81, and a content detailed information area 82. Additionally, the content page 80 may further include a play button 83, a save button 84, a share button 85, etc. The content of the corresponding page 80 may be displayed in the hero area 81.

In this page 80, feedback information related to content playback may be generated, for example, when the user selects the play button 83. Of course, even when the user watches or completes the content, feedback information such as the number of pauses and total playback (viewing) time may be generated.

Additionally, feedback information related to saving and sharing can also be generated through the saving button 84 and the sharing button 88.

So far, examples of recommended content provision methods and user feedback information according to some embodiments of the present disclosure have been described with reference to FIGS. 6 to 8 . According to the above, by providing recommended content through the hero area (e.g., 72) of the content service page, the exposure effect of the recommended content can be maximized and the user's interest and interest can be easily induced.

Hereinafter, various methods that can be performed in the above-described recommendation system 10 will be described with reference to the drawings of FIG. 9 and below. However, in order to provide convenience of understanding, the explanation will be continued assuming that all steps/operations of the methods to be described later are performed in the recommendation system 10 described above. Accordingly, when the subject of a specific step/action is omitted, it can be understood as being performed by the recommendation system 10. However, in a real environment, specific steps/operations of the method to be described later may be performed on another computing device. For example, training on a deep learning model may be performed on other computing devices.

9 is an example flowchart schematically illustrating a machine learning-based recommendation method according to some embodiments of the present disclosure. However, this is only a preferred embodiment for achieving the purpose of the present disclosure, and of course, some steps may be added or deleted as needed.

As shown in FIG. 9, the recommendation method according to embodiments may begin at step S91 of training a deep learning model by performing a next consumption content prediction task using the user's content consumption history data. For example, the recommendation system 10 generates a training set by extracting content sequence samples from content consumption history data and performs a next consumption content prediction task using the training set, thereby performing sequence-to-sequence. You can train a deep learning model based on it. The specific learning method will be explained later along with the structure of the deep learning model. A content sequence may refer to a set of content arranged according to consumption order (i.e., temporal order) and may consist of one or more content.

Content consumption history data may include various information about content consumed by the user without limitation. For example, content consumption history data includes information on consumed content (e.g., title, genre, characters, actors, running time, etc.), information related to consumption/playback of the content (e.g., point of consumption, order of consumption, while playing) Number of pauses, continuous playback time, total playback time, whether played to the end, time taken to play, whether continued viewing when changing devices, time taken to watch until the end, etc.), context-related information (e.g., terminal context (e.g. , terminal type information such as mobile terminal, desktop, TV, etc.), time context (e.g., consumption time information such as weekdays/weekends, time zone, day of the week, etc.), etc.). However, it is not limited to this.

The recommendation system 10 can build a deep learning model for each user, or it can build a deep learning model for customized recommendations for each type of user (e.g., a recommendation model for men in their 20s and a recommendation for women in their 20s) Build each model). The type of user may be classified based on the user's demographic characteristics (e.g., gender, age group, etc.), consumption history, etc. (e.g., types are classified using clustering techniques, etc.), but the scope of the present disclosure is limited to this. It is not limited.

In some cases, the recommendation system 10 may further build a deep learning model for general-purpose recommendation using content consumption history data of multiple users. In addition, the recommendation system 10 uses the content consumption history data of multiple users and the content consumption history data of a specific user (or a specific type of user) together (e.g., combined in an appropriate ratio) to identify a specific user (or a specific type of user). You can also build a deep learning model (i.e., a customized recommendation model) for users. In this case, the cold-start problem can be greatly alleviated because the training set of the deep learning model can be prepared quickly.

The recommendation system 10 may perform various cleaning processes on content consumption history data (or training set) to improve the quality of the training set (or improve the performance of the deep learning model). However, the specific method may vary depending on the embodiment.

In some embodiments, recommendation system 10 may split one content sequence sample into multiple subsequence samples (or extract subsequence samples) based on temporal relevance (or distance), content similarity, etc. ). For example, the recommendation system 10 may extract content consumed within a certain period of time from a content sequence sample as one sub-sequence sample (e.g., content consumed over 3 days is extracted as one sub-sequence sample). As another example, the recommendation system 10 may split the content sequence samples into weekday consumption sequence samples and weekend consumption sequence samples. As another example, the recommendation system 10 determines a point in a content sequence sample where the time distance between two pieces of content is greater than a reference value and divides the content sequence sample into a plurality of sub-sequence samples based on that point. .

In some other embodiments, recommender system 10 may remove content from a sample of content sequences with a consumption (playback) time below a threshold. This is because such content is unlikely to match the user's preferences.

In some other embodiments, the recommendation system 10 may process a long content sequence (hereinafter, may be abbreviated as 'long sequence') sample to an appropriate length. These embodiments will be further described with reference to FIG. 15 later.

In some other embodiments, the recommendation system 10 may remove content (hereinafter, may be abbreviated as 'mega content') whose popularity is above a threshold value from a content sequence sample. These embodiments will be described in more detail later with reference to FIG. 16.

In some other embodiments, data cleaning may be performed based on various combinations of the above-described embodiments.

Hereinafter, the deep learning model 100 and its learning method according to some embodiments of the present disclosure will be described in detail with reference to FIGS. 10 to 13. The deep learning model 100 can be understood as corresponding to the previously mentioned deep learning model (30 in FIG. 3).

10 to 12 are exemplary diagrams for explaining the operation and implementation method of the deep learning model 100 according to some embodiments of the present disclosure.

As shown in FIG. 10, the deep learning model 100 used for content recommendation may be a sequence-to-sequence based model. For example, the deep learning model 100 may be configured to receive a content sequence (e.g., 101, 102) and output a content sequence (e.g., 102, 103). Additionally, the deep learning model 100 may operate to predict individual content constituting the output sequence in an auto-regressive manner. For example, the deep learning model 100 uses the predicted content 102 of the previous step (time) as input to predict the content 103 of the current step (time) (e.g., predicts using the output embedding of the current step) ) can operate to. Of course, in the recommendation stage, if the predicted content (e.g., 102, i.e., next consumption predicted content) and the user's actual consumption content are different, the recommendation system 10 uses the actual consumption content as input instead of the predicted content (e.g., 102). You can also use this to predict the next consumption content. Anyone working in the relevant technical field will already be familiar with the meaning of performing predictions in a 1-received manner, so description of this will be omitted.

The deep learning model 100 described above can be designed/implemented in various ways.

For example, as shown in FIG. 11, the deep learning model 100 may be designed/implemented based only on the decoder among the encoder and decoder of the transformer (see 111, e.g., similar structure to GPT). For example, the deep learning model 100 may be configured to include at least one transformer decoder 111. The deep learning model 100 with this structure can accurately capture and learn the consumption patterns inherent in the content sequence. The transformer decoder 111 may be composed of an attention layer (113, e.g., masked self-attention) and a feedforward layer 112. Those skilled in the art will know the detailed structure and operation of the transformer decoder 111. Since you are probably already familiar with the principle, detailed explanation will be omitted.

As another example, as shown in FIG. 12, the deep learning model 100 may be designed/implemented based on a Recurrent Neural Network (RNN). For example, the deep learning model 100 may be configured to include a plurality of RNN blocks (e.g., 121, 122). The deep learning model 100 with this structure can also accurately capture and learn the consumption patterns inherent in the content sequence.

As another example, the deep learning model 100 may be designed/implemented in a different way from the examples described above.

Below, the structure and learning method of the deep learning model 100 will be described with reference to FIG. 13.

As shown in FIG. 13, the deep learning model 100 may be configured to include an embedding layer 132, an analyzer 131, and a prediction layer 133. In some cases, the deep learning model 100 may be configured to include some additional layers (modules) or to omit some layers (modules).

The embedding layer 132 may refer to a layer (module) that receives information about content (e.g., 134) and generates content embedding (135, e.g., embedding vector). For example, the embedding layer 132 may receive information on individual contents (e.g., 134) constituting the content sequence and generate content embedding (e.g., 135, i.e., content unit embedding). The embedding layer 132 may receive an identification code (e.g., one-hot code) of the content (e.g., 134) and generate a content embedding (e.g., 135) or receive more information to create a richer code. You can also create content embeds. This will be described later with reference to FIG. 14.

For reference, the above-mentioned embedding refers to an expression in an embedding (latent) space and usually has a vector format, so in some cases, it is called 'embedding representation', 'embedding vector', or 'latent representation'. It may also be named as ‘(latent representation)’ or ‘latent vector’. Alternatively, the above-described embedding may be named such as 'embedding code' or latent code.

The embedding layer 132 may be implemented based on a fully-connected layer (or linear) layer, a multi-layer perceptron (MLP), etc. However, the scope of the present disclosure is not limited thereto.

In some embodiments, a code for information on the consumption order of content (i.e., the order/position of the content within the content sequence) may be reflected in the content embedding 135 and input to the analyzer 131. In this case, the analyzer 131 can more accurately determine the consumption order of content, thereby better learning the consumption pattern inherent in the content sequence.

Next, the analyzer 131 is a neural network module that analyzes the embeddings (e.g., 135) of the input content sequence, and can be implemented based on a transformer decoder, RNN block, etc., as described with reference to FIGS. 11 and 12. . For example, the analyzer 131 analyzes the correlation between the input content embeddings (e.g., 135) (e.g., analyzes the correlation between consumed content), synthesizes the analysis results, and analyzes the content embedding (e.g., 135) at each step (time). 136) can be output. The output content embedding (e.g., 136) can be used to perform a predefined prediction task.

Next, the prediction layer 133 may refer to a layer (module) that performs a predefined prediction task. For example, the prediction layer 133 may receive the output content embedding (e.g., 136) of a specific step (point in time) and perform a prediction task. As shown, when the prediction layer 133 is a layer that performs the next consumption content prediction task, the prediction layer 133 may be configured to output a confidence score 137 for each content.

The prediction layer 133 may be implemented based on a fully connected (or linear) layer, MLP, etc. However, the scope of the present disclosure is not limited thereto. The output form of the prediction layer 133 can be appropriately configured to suit the related task.

As described above, the deep learning model 100 can be learned through the next consumption content prediction task. This learning can be performed using content sequence samples included in the training set. For a specific example, the recommendation system 10 inputs (feeds) the output content embedding 136 of the first step (point of view) into the prediction layer 133 to predict the next consumption content (i.e., generates a confidence score 137). calculated), the weight parameters of the deep learning model 100 can be updated (e.g., cross-entropy loss) based on the difference between the prediction result 137 and the correct answer (i.e., the actual content consumed by the user in the following order) Update the prediction layer (133), analyzer (131), and embedding layer (132) as a basis). Additionally, the recommendation system 10 may repeatedly perform the same task for the output content embedding (not shown) of the next step (view). By doing so, the deep learning model 100 has the ability to accurately predict the next consumption content by learning the user's consumption pattern inherent in the content sequence.

For learning efficiency, the recommender system 10 may train the deep learning model 100 through a teacher forcing technique (i.e., inputting the correct answer instead of the prediction result of the previous step). However, the scope of the present disclosure is not limited thereto. Anyone working in the relevant technical field will already be familiar with the teacher forcing technique, so detailed explanations thereof will be omitted.

Meanwhile, in some embodiments, the recommendation system 10 further performs the task of predicting the attributes (characteristics) (e.g., genre, character, actor, background music, running time, etc.) of the next consumed content to create a deep learning model. (100) can also be learned. For example, the recommendation system 10 may further perform these tasks through a separate prediction layer configured to output confidence scores for each attribute (characteristic). In this case, the performance of the deep learning model 100 can be further improved.

So far, the deep learning model 100 and its learning method according to some embodiments of the present disclosure have been described with reference to FIGS. 10 to 13.

Description will be made again with reference to FIG. 9 .

In step S92, information about recent content sequences consumed by the user may be obtained. For example, the recommendation system 10 may obtain information about at least one content that the user recently consumed. Recently consumed content may refer, for example, to content consumed within a certain period of time based on the point of recommendation.

In some embodiments, the recommender system 10 may perform the above-described purification processing (e.g., removal of mega content, removal of content with a short consumption time, length processing of long sequences, etc.) on the user's recent content sequence. By doing so, recommendation accuracy can be further improved.

In step S93, recommended content may be determined by predicting the next consumed content from the recent content sequence through a learned deep learning model. For example, the recommendation system 10 can predict the next content to be consumed by inputting a recent content sequence into a deep learning model and determine recommended content based on the predicted content. However, the specific method may vary depending on the embodiment.

In some embodiments, the recommendation system 10 may calculate confidence scores for a plurality of pre-registered contents through a prediction layer of a deep learning model. Additionally, the recommendation system 10 may determine at least one content for which the calculated confidence score is greater than or equal to the reference value as recommended content.

In some other embodiments, the recommendation system 10 may repeatedly perform the prediction process for the next consumption content in an autoregressive manner using a deep learning model. For example, the recommendation system 10 can predict the next consumption content at the current point (step) by using the predicted content at the previous point (step) as an input to a deep learning model, and repeat this process a preset number of times. You can. As a result, a plurality of recommended candidate contents (i.e., predicted contents) may be derived. Additionally, the recommendation system 10 may determine recommended content from among a plurality of recommended candidate content.

In some other embodiments, the recommendation system 10 predicts the next consumption content through a deep learning model and matches the predicted content with similar content (e.g., content featuring the same actor, content with similar background music, and the same genre). content, etc.) can be determined as recommended content.

In some other embodiments, recommended content may be determined based on various combinations of the above-described embodiments.

In some cases, the recommendation system 10 predicts the properties (characteristics) of the next consumption content (e.g., genre, characters, actors, running time, background music, etc.) through a deep learning model and determines the same or similar properties (characteristics) The contents of may be determined as recommended content.

Meanwhile, the recommendation system 10 may determine recommended content periodically or aperiodically.

For example, the recommendation system 10 may determine new recommended content according to a preset time period. If the user consumes additional content before the recommendation time arrives, the recommendation system 10 may determine the recommended content at the current time using the recent content sequence including the additional content (i.e., the recent content sequence is also (may be updated periodically). In the opposite case, the recommendation system 10 may further use the recommended content at a previous point in time (e.g., predicted next consumption content) as input (i.e., in an autoregressive manner) to determine the recommended content at the current point in time. . Alternatively, the recommendation system 10 may determine recommended content at the current time by reusing the most recent content sequence at a previous time.

As another example, recommendation system 10 may update (i.e., re-determine) recommended content in response to an event, such as a user consuming additional content.

As another example, the recommendation system 10 may determine recommended content based on various combinations of the examples described above.

Meanwhile, according to some embodiments of the present disclosure, the recommendation system 10 may rebuild the deep learning model periodically or aperiodically. For example, the recommendation system 10 may regenerate a training set based on recent content consumption history data to reflect the user's recent preferences (e.g., historical data older than a certain period of time is deleted). And, the recommender system 10 can rebuild the deep learning model using the generated training set (e.g., retraining after initialization). As another example, when the user's negative feedback information about recommended content is continuously received (e.g., when the content is not selected more than a preset number of times), the recommendation system 10 may determine that the user's preference has changed. Additionally, the recommendation system 10 can rebuild the deep learning model using recent content consumption history data. As another example, the recommendation system 10 reconfigures the prediction layer of the deep learning model (e.g., adds nodes corresponding to new content) to reflect new content in the prediction results of the deep learning model and rebuilds the deep learning model. can do. In some cases, the recommendation system 10 may change only the nodes of the prediction layer corresponding to new content to the active state without rebuilding the deep learning model, which will be described later with reference to FIG. 17. As another example, the recommender system 10 reconfigures the prediction layer of the deep learning model to remove terminated content from the prediction results of the deep learning model (e.g., reorganizes it to fit the currently registered content) and reorganizes the deep learning model again. It can be built (or the node corresponding to the end content can be changed to a disabled state).

Additionally, according to some embodiments, the recommendation system 10 may perform content recommendation using a reinforcement learning model and a deep learning model together. For example, the recommendation system 10 may determine the recommended content through a reinforcement learning model under the first condition (situation) and determine the recommended content through a deep learning model under the second condition (situation), which is explained in FIG. 18. Please refer to the contents further.

Additionally, according to some embodiments, the recommendation system 10 may train a deep learning model using user history data collected from different domains and perform multi-domain recommendation through the learned deep learning model. , This will be described later with reference to FIGS. 20 to 23.

So far, a machine learning-based recommendation method according to some embodiments of the present disclosure has been described with reference to FIGS. 9 to 13 . According to the above, a sequence-to-sequence based deep learning model can be learned by performing a next consumption content prediction task using content sequence samples extracted from the user's content consumption history data. In this case, content recommendation accuracy can be greatly improved because the deep learning model can accurately predict the next consumed content by capturing the sequential consumption patterns inherent in the user's content consumption history.

The technical ideas of the embodiments described so far can be applied to items in other domains (types). For example, a product recommendation function can be implemented through a sequence-to-sequence-based deep learning model that learns the user's shopping history data, and a sequence-to-sequence-based deep learning model that learns the user's order history data Through this, a food recommendation function can be implemented.

Hereinafter, a content embedding method according to some embodiments of the present disclosure will be described with reference to FIG. 14.

As shown in FIG. 14, the recommendation system 10 may generate the final embedding 148 for the content 141 based on various information (e.g., 142 to 144) of the content 141.

For example, the recommendation system 10 generates the first embedding 145 from the identification code 142 (e.g., one-hot code) of the content 141 (e.g., generated by passing the embedding layer), and the title The second embedding 146 can be generated from text information such as (143) (e.g., generated using an appropriate text embedding technique), and the third embedding 147 can be generated from the representative image 144 (e.g. , generated using an appropriate image embedding technique). Additionally, the recommendation system 10 may generate content embedding 148 by aggregating the embeddings 145 to 147 for each information. Here, aggregating may be performed through operations such as concatenation, addition, multiplication, element-by-element multiplication, etc., or may be performed in other ways (e.g., aggregating through a neural network layer such as MLP).

In some cases, the recommendation system 10 further aggregates (reflects) other attribute (characteristic) information (e.g., genre, character, actor, background music, running time, etc.) in addition to the title to the content embedding 148 to determine the content. We can also generate the final content embedding for (141). For example, the recommendation system 10 may generate the final content embedding by aggregating the content embedding 148 with embeddings for attribute (characteristic) information.

Additionally, the recommendation system 10 may further aggregate (reflect) the user's context information (e.g., terminal context, time context, etc.) to the content embedding 148 to generate the final content embedding for the content 141. . For example, the recommendation system 10 may generate an embedding for the user's context information through an appropriate embedding technique, and generate a final content embedding by aggregating the generated embedding and the content embedding 148. In this case, the deep learning model predicts the next consumption content by considering the user's context information, so the performance of the deep learning model can be further improved.

So far, a content embedding method according to some embodiments of the present disclosure has been described with reference to FIG. 14. According to the above, content embedding can be created based on various information of the content (e.g., identification code, title, representative image, etc.). In this case, the deep learning model is trained to predict the next consumption content by comprehensively considering various information about the contents, so the recommendation accuracy of the deep learning model can be further improved.

Hereinafter, embodiments of the data purification method will be described with reference to FIGS. 15 and 16.

FIG. 15 is an exemplary diagram for explaining a data purification method according to some embodiments of the present disclosure.

As shown in FIG. 15, these embodiments relate to a method of improving the quality of a training set by processing long-length content sequence (i.e., long sequence) samples into shorter lengths. The reason for performing this processing (or the reason why the quality of the training set is improved) is that long sequence samples inevitably contain significant noise (e.g., content that does not match the user's content consumption pattern or preference). In other words, these long sequence samples may cause distorted (mis)learning of the user's content consumption pattern, and as a result, the performance of the deep learning model 100 may actually deteriorate.

Specifically, the recommendation system 10 may identify a long sequence sample 151 whose length is greater than or equal to a reference value among content sequence samples. Additionally, the recommendation system 10 may divide the long sequence sample 151 into sub-sequence samples 152 and 153 with a length less than the reference value. For example, the recommendation system 10 may extract a plurality of sub-sequence samples 152 and 153 from the long sequence sample 151. Here, division (or extraction) of the sequence may or may not be performed in such a way that overlapping content (e.g., C) exists between the subsequence samples 152 and 153.

The above-described data purification method can also be applied at the inference stage (i.e., the actual recommendation stage). For example, when the length of the most recent content sequence is greater than the standard value, the recommendation system 10 extracts a content sequence (e.g., most recently consumed content) with a length less than the standard value from the sequence and uses it to select the next consumed content. It can also be predicted.

Hereinafter, a data purification method according to some other embodiments of the present disclosure will be described with reference to FIG. 16. In Figure 16, the size of the shape indicates the popularity of the corresponding content.

As shown in FIG. 16, these embodiments relate to a method of improving the quality of the training set by removing content (i.e., mega content) whose popularity is above a standard value. The reason why this processing is performed (or the quality of the training set is improved) is because it is difficult to view mega content as content that reflects the preferences (tastes) of individual users. In other words, these mega contents may also cause distorted (mis)learning of the user's content consumption pattern, and as a result, the performance of the deep learning model 100 may deteriorate.

Specifically, the recommendation system 10 may remove the content 162 having a popularity higher than the reference value from the content sequence sample 161 (see 163). Even through this simple removal process, the quality of the training set can be greatly improved.

In some cases, the recommendation system 10 may solve problems related to mega content through weighted learning according to popularity. For example, the recommendation system 10 adjusts the prediction loss (e.g., cross-entropy loss) by reflecting the weight according to popularity (e.g., reflects a low weight in the prediction loss of mega content to reduce the loss value, and vice versa. The loss value is amplified by reflecting the high weight), and a deep learning model (e.g., 100) can be trained based on the adjusted prediction loss.

The above-described data purification method can also be applied at the inference stage (i.e., the actual recommendation stage). For example, the recommendation system 10 may remove mega content from the user's recent content sequence and use the remaining sequence to predict the next content to be consumed. By doing so, recommendation accuracy can be further improved.

So far, embodiments of the data purification method have been described with reference to FIGS. 15 and 16. Below, with reference to FIG. 17, problems caused by new content registration and methods for solving them will be explained.

In content-related domains, new content is continuously created and registered. However, because the prediction layer of the existing deep learning model is not configured to output confidence scores for new content, a problem may occur in which new content is completely excluded from the prediction results of the deep learning model.

In some embodiments of the present disclosure, in order to solve the above problem, extra nodes (e.g., 174 to 176) may be allocated in advance to the prediction layer 170 of the deep learning model. Additionally, nodes 171 to 173 corresponding to pre-registered content (see A to C) may be set to an activated state, and extra nodes (e.g., 174 to 176) may be set to a deactivated state. Here, the fact that a node is in an inactive state may mean that the output value (e.g., confidence score of the corresponding content) of the node is not used during learning and/or inference.

As shown on the right side of FIG. 17, when new content (see D) is registered, the recommendation system 10 may change the node 174 corresponding to the new content to the activated state. Additionally, the recommendation system 10 can additionally learn a deep learning model using the user's consumption history data for new content and determine recommended content using the deep learning model. In this case, reconstruction (e.g., retraining after initialization) of the deep learning model does not need to be performed, so the time cost and computing cost required for model learning can be greatly reduced.

So far, with reference to FIG. 17, problems caused by new content registration and ways to solve them have been explained. Hereinafter, a method of operating a recommendation model according to some embodiments of the present disclosure will be described with reference to FIG. 18. The reinforcement learning model 180 illustrated in FIG. 18 can be understood as corresponding to the reinforcement learning model (20 in FIG. 2) described above.

As shown in FIG. 18, these embodiments relate to a method of performing recommendation using the reinforcement learning model 180 and the deep learning model 100 together. For example, the recommendation system 10 may perform recommendation using the reinforcement learning model 180 under the first condition (situation) and use the deep learning model 100 under the second condition (situation). . Of course, in some cases, the recommendation system 10 may perform recommendations by using the prediction results of the two models 100 and 180 together. For reference, Figure 18 illustrates a specific scenario in which two recommendation models 100 and 180 are operated, but the scope of the present disclosure is not limited to this example.

For a specific example, when the amount of content consumption history data for a user is less than the standard value (e.g., at the beginning of recommendation when the history data is not sufficiently accumulated), the recommendation system 10 uses the reinforcement learning model 180 to make recommendations. It can be performed (refer to T1-T2 section). In some cases, the recommendation system 10 may randomly determine recommended content without using the reinforcement learning model 180.

In addition, for example, if the amount of content consumption history data for the user is greater than the standard value, the recommendation system 10 can perform recommendations using a deep learning model (100, that is, a deep learning model that learned the data). Yes (refer to T2-T3 section). For example, the recommendation system 10 may change the recommendation model from the reinforcement learning model 180 to the deep learning model 100.

Additionally, for example, the recommender system 10 may rebuild the deep learning model 100 periodically or aperiodically and perform recommendations (see section T3-T4). While the deep learning model 100 is being rebuilt, the recommendation system 10 may use the reinforcement learning model 180 or may randomly determine recommended content.

In addition, for example, when the number of registered new contents is greater than the reference value, the recommendation system 10 reconfigures the prediction layer of the deep learning model 100 and collects the user's consumption history data for the new contents. . Additionally, the recommendation system 10 can rebuild the deep learning model 100 by performing re-learning based on the collected consumption history data (refer to the section after T5). This reconstruction may mean building a new deep learning model. While the deep learning model 100 is being rebuilt, the recommendation system 10 may use the reinforcement learning model 180 or randomly determine recommended content (see section T4-T5).

So far, a method of operating a recommendation model according to some embodiments of the present disclosure has been described with reference to FIG. 18 . According to the above, by operating the reinforcement learning model 180 and the deep learning model 100 in a complementary manner, user satisfaction with recommended content can be maintained at a continuously high level.

Hereinafter, a machine learning-based multi-domain recommendation method according to some embodiments of the present disclosure will be described with reference to FIGS. 19 to 23.

As shown in FIG. 19 and the like, these embodiments relate to a method of performing multi-domain (or cross-domain) recommendation using the deep learning model 190. First, the structure, item embedding method, and learning method of the deep learning model 190 will be explained.

As shown in FIG. 19, the deep learning model 190 may be configured to receive a multi-domain item sequence (e.g., 191) and output a multi-domain item sequence (e.g., 192). Additionally, the deep learning model 190 may also operate to predict individual items (e.g., 192) constituting the output sequence in an autoregressive manner. The deep learning model 190 may be configured with a structure similar to the deep learning model 100 described above.

Meanwhile, the specific method of generating the embedding of the item (e.g., 191) may vary depending on the embodiment.

In some embodiments, an embedding of an item may be generated (e.g., generated by passing it through an embedding layer) based on a code that can uniquely identify the item across multiple domains (e.g., a one-hot code). . This identification code may be defined based on a combination of a domain identification code and an item identification code or may be defined in other ways.

In some other embodiments, as shown in FIG. 20, the final embedding of the item 201 is achieved by aggregating (e.g., concatenation, addition, multiplication, element-by-element multiplication, etc.) the embeddings 202 to 204 of each domain. (205) can be generated. For example, let's assume that the target item 201 is an item of the first domain (see D1). In this case, the recommendation system 10 may generate the embedding 202 of the item 201 in a manner suitable for the item of the first domain (e.g., for content, refer to the description of FIG. 14, etc.). And, the recommendation system 10 generates the final embedding 205 by aggregating the embedding 202 of the first domain and the embeddings 203, 204, e.g., null vectors of the corresponding item 201 in other domains. You can.

In some other embodiments, as shown in Figure 21, the domain identification code (e.g., one-hot code) is reflected in the embedding (e.g., 213, 214) of each item (e.g., 212, 213) It can be input to the analyzer 211 of the deep learning model 190. For example, the recommendation system 10 reflects the identification code of the first domain in the embedding 213 of the item 212 belonging to the first domain (see D1) (e.g., concatenation, addition, multiplication, multiplication by element, etc.) It can be input (feeding) to the analyzer 211. Similarly, the recommendation system 10 may reflect the identification code of the second domain in the embedding 214 of the item 213 belonging to the second domain (see D2) and input (feed) it to the analyzer 211. By doing so, the analyzer 211 can accurately determine the domain of the input item embedding (e.g., 213, 214) and better learn the user's behavior (consumption) pattern.

In some other embodiments, an embedding of an item may be created based on various combinations of the above-described embodiments.

Meanwhile, the prediction layer of the deep learning model 190 may be configured to perform only the next consumption item prediction task for a specific domain or may be configured to perform the next consumption item prediction task for multiple domains.

For example, as shown in FIG. 22, the prediction layer 221 of the deep learning model 190 may be configured to perform the task of predicting the next consumption item for a multi-domain. That is, the prediction layer 221 may be configured to output the confidence score 224 of items belonging to a multi-domain (e.g., all items). Specifically, the prediction layer 221 receives the item embedding 223 (e.g., output at each time point (step)) output from the analyzer 211 and outputs a confidence score 224 for items belonging to the multi-domain. can do. As described above, the analyzer 211 may analyze the embeddings (e.g., 223) of the items (e.g., 222) constituting the input sequence and output the item embeddings (e.g., 223) at every time point (step).

As another example, the deep learning model 190 includes a first prediction layer that performs a next consumption item prediction task for the first domain and a second prediction layer that performs a next consumption item prediction task for the second domain. It can be.

As another example, as shown in FIG. 23, the deep learning model 190 includes a plurality of prediction layers 232 to 234 that perform the next consumption item prediction task of different domains and the next domain prediction task. It may be configured to include a layer 231. At this time, the domain prediction layer 231 may be configured to output a confidence score 237 for each domain, and each of the prediction layers 232 to 234 may output a confidence score 238-1 to 238-3 for each item of the domain. It can be configured to output. In addition, all prediction layers 231 to 234 may receive the output embedding (e.g., 236) for the item (e.g., 235) and perform a prediction task.

The deep learning model 190 described above can be learned through the next consumption item prediction task, the next domain prediction task, etc. For example, the recommendation system 10 may generate a training set by extracting a plurality of item sequence samples from multi-domain consumption history data. At this time, the item sequence sample may include items from multiple domains. Additionally, the recommendation system 10 can learn the deep learning model 190 by performing a next consumption item prediction task using the training set. Since the specific learning process is similar to the deep learning model 100 described above, further explanation will be omitted.

Once learning is complete, the recommendation system 10 can provide a multi-domain recommendation service using the learned deep learning model 190. Specifically, the recommendation system 10 may determine a recommended item by inputting the user's recent item sequence into the deep learning model 190 to predict the next consumption item. However, the specific method may vary depending on the embodiment.

In some embodiments, the recommendation system 10 may determine items with confidence scores greater than or equal to a reference value as recommended items without distinction of domains. For example, referring again to FIG. 22 , the recommendation system 10 may compare the confidence scores 224 of multi-domain items output from the prediction layer 221 and select one or more items having a confidence score greater than a reference value. Additionally, the recommendation system 10 may determine the selected item as a recommended item.

In some other embodiments, the recommendation system 10 may repeatedly perform the prediction process for the next consumption item in an autoregressive manner using the deep learning model 190. Additionally, the recommendation system 10 may determine a recommended item from among a plurality of items predicted through this iterative process.

In some other embodiments, the recommendation system 10 may determine a recommended item for each domain. For example, referring again to FIG. 23, the recommendation system 10 determines recommended items of the first domain based on the confidence scores 238-1 of the items output from the first prediction layer 232 and A recommended item of the second domain may be determined based on the confidence score 238-2 of the items output at 233. Similarly, the recommendation system 10 may also determine recommended items of the third domain based on the confidence scores 238-3 of the items output from the third prediction layer 234.

In some other embodiments, the recommendation system 10 may predict the user's next domain and determine a recommended item from among the items in the predicted domain. For example, referring again to FIG. 23, the recommendation system 10 can predict the next domain through the domain prediction layer 231 (e.g., refer to the D2 domain). Additionally, the recommendation system 10 may determine a recommended item based on the confidence score 238-2 of the items belonging to the predicted domain. By doing so, recommendation accuracy can be further improved.

In some other embodiments, a recommended item may be determined based on various combinations of the above-described embodiments.

So far, a machine learning-based multi-domain recommendation method according to some embodiments of the present disclosure has been described with reference to FIGS. 19 to 23. According to the above, a multi-domain recommendation service can be provided through a deep learning model learned using consumption history data of multi-domains. For example, a service that recommends delivery food or products to a user consuming content may be provided. In this case, a new (fresh) experience with the recommended service may be provided to the user. In addition, because the consumption history data required for model learning can be quickly prepared, the cold-start problem, one of the chronic problems in the recommendation field, can be easily solved.

Hereinafter, an exemplary computing device 240 capable of implementing the recommendation system 10 according to some embodiments of the present disclosure will be described with reference to FIG. 24.

FIG. 24 is an exemplary hardware configuration diagram showing the computing device 240.

As shown in FIG. 24, the computing device 240 includes one or more processors 241, a bus 243, a communication interface 244, and a memory (loading) a computer program executed by the processor 241. 242) and a storage 247 that stores a computer program 246. However, only components related to the embodiment of the present disclosure are shown in FIG. 24. Accordingly, a person skilled in the art to which this disclosure pertains can see that other general-purpose components may be included in addition to the components shown in FIG. 24. That is, the computing device 240 may further include various components in addition to those shown in FIG. 24 . Additionally, in some cases, the computing device 240 may be configured with some of the components shown in FIG. 24 omitted. Hereinafter, each component of the computing device 240 will be described.

The processor 241 may control the overall operation of each component of the computing device 240. The processor 241 may be a Central Processing Unit (CPU), Micro Processor Unit (MPU), Micro Controller Unit (MCU), Graphic Processing Unit (GPU), Neural Processing Unit (NPU), or any other device well known in the art of the present disclosure. It may be configured to include at least one of the following types of processors. Additionally, the processor 241 may perform operations on at least one application or program to execute operations/methods according to embodiments of the present disclosure. Computing device 240 may include one or more processors.

Next, memory 242 may store various data, instructions and/or information. Memory 242 may load a computer program 246 from storage 247 to execute operations/methods according to embodiments of the present disclosure. The memory 242 may be implemented as a volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

Next, bus 243 may provide communication functionality between components of computing device 240. The bus 243 may be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

Next, the communication interface 244 may support wired and wireless Internet communication of the computing device 240. Additionally, the communication interface 244 may support various communication methods other than Internet communication. To this end, the communication interface 244 may be configured to include a communication module well known in the technical field of the present disclosure.

Next, storage 247 may non-transitory store one or more computer programs 246. The storage 247 may be used in a non-volatile memory such as Read Only Memory (ROM), Erasable Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), flash memory, a hard disk, a removable disk, or the like in the technical field to which this disclosure pertains. It may be configured to include any known type of computer-readable recording medium.

Next, the computer program 246 may include one or more instructions that, when loaded into the memory 242, cause the processor 241 to perform operations/methods according to various embodiments of the present disclosure. That is, the processor 241 can perform operations/methods according to various embodiments of the present disclosure by executing the one or more instructions.

For example, the computer program 246 performs a next consumption content prediction task using the user's content consumption history data to obtain a learned sequence-to-sequence based deep learning model, the sequence of content consumed by the user It may include instructions for performing an operation of obtaining information about and determining recommended content for the user by predicting the next consumption content from the content sequence through a learned deep learning model. In this case, the recommendation system 10 according to some embodiments of the present disclosure may be implemented through the computing device 240.

Meanwhile, in some embodiments, the computing device 240 shown in FIG. 24 may mean a virtual machine implemented based on cloud technology. For example, the computing device 240 may be a virtual machine running on one or more physical servers included in a server farm. In this case, at least some of the processor 241, memory 242, and storage 247 shown in FIG. 24 may be virtual hardware, and the communication interface 244 may also be a virtual switch, etc. It may be implemented as a virtualized networking element.

So far, an exemplary computing device 240 capable of implementing the recommendation system 10 according to some embodiments of the present disclosure has been described with reference to FIG. 24 .

So far, various embodiments of the present disclosure and effects according to the embodiments have been mentioned with reference to FIGS. 1 to 24 . The effects according to the technical idea of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below.

In addition, although it has been described in the above embodiments that a plurality of components are combined or operated in combination, the technical idea of the present disclosure is not necessarily limited to these embodiments. That is, as long as it is within the scope of the technical idea of the present disclosure, all of the components may be operated by selectively combining one or more of them.

The technical ideas of the present disclosure described so far can be implemented as computer-readable code on a computer-readable medium. A computer program recorded on a computer-readable recording medium can be transmitted to another computing device through a network such as the Internet, installed on the other computing device, and thus used on the other computing device.

Although operations are shown in the drawings in a specific order, it should not be understood that the operations must be performed in the specific order shown or sequential order or that all illustrated operations must be performed to obtain the desired results. In certain situations, multitasking and parallel processing may be advantageous. Although various embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will understand that the technical idea of the present disclosure can be modified in a different specific form without changing the technical idea or essential features. It is understandable that it can also be implemented. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. The scope of protection of this disclosure should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of rights of the technical ideas defined by this disclosure.

Claims (19)

A method performed by at least one computing device, comprising:
Obtaining a sequence-to-sequence based deep learning model learned using the user's content consumption history data - the deep learning model was learned through a next consumption content prediction task -;
Obtaining information about the content sequence consumed by the user; and
Determining recommended content for the user by predicting next consumed content from the content sequence through the deep learning model,
The learning process of the deep learning model is:
extracting a plurality of content sequence samples from the content consumption history data;
generating a training set for the next consumption content prediction task by removing content whose popularity is higher than a threshold value from a specific content sequence sample among the plurality of content sequence samples; and
Comprising training the deep learning model by performing the next consumption content prediction task using the training set,
Machine learning-based recommendation method. According to paragraph 1,
The recommended content is provided to the user through the hero area of the content service page,
Machine learning-based recommendation method. According to paragraph 1,
The deep learning model is implemented based only on the decoder among the encoder and decoder of the transformer,
Machine learning-based recommendation method. According to paragraph 1,
The step of generating a training set for the next consumption content prediction task is,
Selecting a content sequence sample whose length is equal to or greater than a reference value from among the plurality of content sequence samples; and
Generating the training set for the next consumed content prediction task by dividing the selected content sequence samples into subsequence samples with a length less than the reference value,
Machine learning-based recommendation method. delete According to paragraph 1,
The step of determining the recommended content is,
generating content embedding for the individual content based on the identification code of the individual content included in the content sequence; and
Including the step of predicting the next consumption content by inputting the content embedding into the deep learning model,
Machine learning-based recommendation method. According to paragraph 1,
The step of determining the recommended content is,
generating a first embedding based on text associated with individual content included in the content sequence;
generating a second embedding based on an image associated with the individual content;
generating a content embedding for the individual content by aggregating the first embedding and the second embedding; and
Including the step of predicting the next consumption content by inputting the content embedding into the deep learning model,
Machine learning-based recommendation method. According to paragraph 1,
The step of determining the recommended content is,
generating a first content embedding based on information on individual content included in the content sequence;
generating a second content embedding by reflecting the user's context information associated with the individual content in the first content embedding; and
Including the step of predicting the next consumption content by inputting the second content embedding into the deep learning model,
Machine learning-based recommendation method. According to paragraph 1,
The step of determining the recommended content is,
Comprising the step of determining the recommended content by removing content whose popularity is above a threshold value from the content sequence and predicting the next consumed content using the remaining content sequence.
Machine learning-based recommendation method. According to paragraph 1,
The deep learning model includes a prediction layer configured to output a confidence score for each content,
The step of determining the recommended content is,
calculating a confidence score for a plurality of pre-registered contents through the prediction layer; and
Comprising the step of determining a content whose confidence score is equal to or higher than a standard value among the plurality of content as the recommended content.
Machine learning-based recommendation method. According to paragraph 1,
The step of determining the recommended content is,
deriving a plurality of candidate contents by repeatedly predicting the next consumption content in an auto-regressive manner using the deep learning model; and
Comprising the step of determining the recommended content among the plurality of candidate content,
Machine learning-based recommendation method. According to paragraph 1,
The step of determining the recommended content is,
When the amount of content consumption history data is greater than a reference value, determining the recommended content through the deep learning model that learned the content consumption history data,
If the amount of content consumption history data is less than the standard value, the recommended content is determined through a reinforcement learning model that learned the user's feedback,
Machine learning-based recommendation method. A method performed by at least one computing device, comprising:
Obtaining a sequence-to-sequence based deep learning model learned using the user's item consumption history data - the deep learning model was learned through a next consumption item prediction task -;
Obtaining information about the sequence of items consumed by the user; and
Determining a recommended item for the user by predicting the next consumption item from the item sequence through the deep learning model,
The learning process of the deep learning model is:
extracting a plurality of item sequence samples from the item consumption history data;
generating a training set for the next consumption item prediction task by removing an item whose popularity is higher than a reference value from a specific item sequence sample among the plurality of item sequence samples; and
Comprising training the deep learning model by performing the next consumption item prediction task using the training set,
Machine learning-based recommendation method. According to clause 13,
The item consumption history data includes item consumption history data of a first domain and item consumption history data of a second domain,
At least some of the plurality of item sequence samples include items of the first domain and items of the second domain,
Machine learning-based recommendation method. According to clause 14,
The step of determining the recommended item is,
predicting a first next consumption item belonging to the first domain and a second next consumption item belonging to the second domain through the deep learning model; and
Comprising the step of determining the recommended item among the first next consumption item and the second next consumption item,
Machine learning-based recommendation method. According to clause 14,
The step of determining the recommended item is,
generating item embeddings based on information on individual items included in the item sequence; and
Comprising the step of reflecting the domain identification code of the individual item in the item embedding and inputting it into the deep learning model to predict the next consumption item,
Machine learning-based recommendation method. According to clause 13,
The item consumption history data includes item consumption history data of a first domain and item consumption history data of a second domain,
The item sequence includes items of the first domain,
The step of determining the recommended item is,
Comprising determining the recommended item by predicting the next consumption item of the second domain through the deep learning model,
Machine learning-based recommendation method. One or more processors; and
comprising a memory that stores a computer program executed by the one or more processors,
The computer program:
An operation of acquiring a sequence-to-sequence-based deep learning model learned using the user's content consumption history data - the deep learning model is learned through a next consumption content prediction task -;
Obtaining information about a content sequence consumed by the user; and
Includes instructions for determining recommended content for the user by predicting next consumption content from the content sequence through the deep learning model,
The learning process of the deep learning model is:
extracting a plurality of content sequence samples from the content consumption history data;
generating a training set for the next consumption content prediction task by removing content whose popularity is higher than a threshold value from a specific content sequence sample among the plurality of content sequence samples; and
Comprising the operation of training the deep learning model by performing the next consumption content prediction task using the training set,
Recommendation system. One or more processors; and
comprising a memory that stores a computer program executed by the one or more processors,
The computer program:
An operation of acquiring a sequence-to-sequence based deep learning model learned using the user's item consumption history data - the deep learning model is learned through a next consumption item prediction task -;
Obtaining information about a sequence of items consumed by the user; and
Includes instructions for determining a recommended item for the user by predicting the next consumption item from the item sequence through the deep learning model,
The learning process of the deep learning model is:
extracting a plurality of item sequence samples from the item consumption history data;
generating a training set for the next consumption item prediction task by removing an item whose popularity is higher than a threshold value from a specific item sequence sample among the plurality of item sequence samples; and
Including the operation of training the deep learning model by performing the next consumption item prediction task using the training set,
Recommendation system.

Images