CN116629944A

CN116629944A - Recommendation method, system and storage medium

Info

Publication number: CN116629944A
Application number: CN202310676249.3A
Authority: CN
Inventors: 张进; 李逍遥; 张晓敏
Original assignee: China United Network Communications Group Co Ltd
Current assignee: China United Network Communications Group Co Ltd
Priority date: 2023-06-07
Filing date: 2023-06-07
Publication date: 2023-08-22

Abstract

The application provides a recommendation method, a recommendation system and a storage medium, relates to the technical field of computers, and can solve the problems of high time delay and low efficiency of a recommendation system in the prior art. The recommendation system includes: recommending an offline recommendation result to the user in response to a login operation of the user; the offline recommendation result is determined according to offline user feature data and an offline user feature model, wherein the offline user feature data is obtained according to an Impala algorithm, and the offline user feature model is constructed according to a PySprak framework; responding to a preset operation of a user, and determining a user behavior log; determining a real-time recommendation result according to the user behavior log; and displaying the offline recommendation result and the real-time recommendation result to the user according to the front-end service. The recommendation method and the recommendation system are used for recommending to the user according to the user preference when the user uses the recommendation system.

Description

Recommendation method, system and storage medium

Technical Field

The present application relates to the field of computer technologies, and in particular, to a recommendation method, a recommendation system, and a storage medium.

Background

The recommendation system at the present stage classifies the architecture into three layers: a real-time service layer (online), a near real-time service layer (online), an offline service layer (offline). Wherein real-time services have high latency requirements in order to quickly respond to user requests and to enable interactions with users. The near real-time service layer cannot update real-time recommendation results in real time. Offline services cannot update data with very low time delay and realize real-time updating of models.

Summarizing, in the current recommended system architecture, there is still a problem that lower latency and more data processing cannot be achieved, and efficiency is low.

Disclosure of Invention

The application provides a recommendation method, a recommendation system and a storage medium, which are used for solving the problems of high time delay and low efficiency of the recommendation system in the current stage.

In order to achieve the above purpose, the application adopts the following technical scheme:

in a first aspect, the present application provides a recommendation method, including: recommending an offline recommendation result to the user in response to a login operation of the user; the offline recommendation result is determined according to offline user feature data and an offline user feature model, wherein the offline user feature data is obtained according to an Impala algorithm, and the offline user feature model is constructed according to a PySprak framework; responding to a preset operation of a user, and determining a user behavior log; determining a real-time recommendation result according to the user behavior log; the real-time recommendation result is determined according to the real-time user characteristic model and the user real-time characteristic data, and the user real-time characteristic data is determined according to the user behavior log; and displaying the offline recommendation result and the real-time recommendation result to the user according to the front-end service.

In one possible implementation manner, the method further includes: and updating the real-time user characteristic model according to the Flink algorithm.

In one possible implementation, the offline recommendation results and the real-time recommendation results are stored in a Redis database.

In one possible implementation, the user behavior log is stored in Kafka middleware.

In a second aspect, the present application provides a recommendation device, the device comprising: a processing unit; the processing unit is used for responding to the login operation of the user and recommending an offline recommendation result to the user; the offline recommendation result is determined according to offline user feature data and an offline user feature model, wherein the offline user feature data is obtained according to an Impala algorithm, and the offline user feature model is constructed according to a PySprak framework; the processing unit is further used for responding to preset operation of the user and determining a user behavior log; the processing unit is also used for determining a real-time recommendation result according to the user behavior log; the real-time recommendation result is determined according to the real-time user characteristic model and the user real-time characteristic data, and the user real-time characteristic data is determined according to the user behavior log; and displaying the offline recommendation result and the real-time recommendation result to the user according to the front-end service.

In a possible implementation, the processing unit is specifically further configured to: and updating the real-time user characteristic model according to the Flink algorithm.

In one possible implementation, the recommendation device further includes: a storage unit; and the storage unit is used for storing the offline recommendation result and the real-time recommendation result in the Redis database.

In a possible implementation, the storage unit is further configured to store the user behavior log in Kafka middleware.

In a third aspect, the present application provides a recommendation system, comprising: an offline service layer, a near real-time service layer and a real-time service layer; the offline service layer is used for determining an offline recommendation result according to the offline feature data of the user and the offline user feature model; the offline user characteristic data are obtained according to an Impala algorithm, and the offline user characteristic model is constructed according to a PySprak framework; the near real-time service layer is used for determining real-time recommendation results according to the real-time characteristic data of the user; the real-time service layer is used for responding to the user request and acquiring a user behavior log; the user behavior log is used for determining the real-time characteristic data of the user.

In one possible implementation, the offline service layer is further configured to obtain offline data from the distributed file system HDFS according to an Impala algorithm; the offline service layer is also used for carrying out user characteristic extraction and data preprocessing on the offline data according to an Impala algorithm and determining the offline characteristic data of the user; and the offline service layer is also used for determining an offline user characteristic model according to the Pyspark framework and the user offline characteristic data.

In one possible implementation, the near real-time service layer is further configured to update the real-time user feature model according to the flank algorithm; the real-time user characteristic model is used for determining real-time recommendation results by the recommendation system.

In one possible implementation, the recommendation system further includes: a Redis database; the Redis database is used for storing data of the offline service layer, the near real-time service layer and the real-time service layer in operation;

in one possible implementation, the real-time service layer includes a Kafka middleware; kafka middleware for storing user behavior logs.

In one possible implementation, the offline service layer is further configured to perform task scheduling according to an airlow algorithm; the task scheduling is used for indicating that the offline recommendation result is determined at least one preset time point.

In one possible implementation, the offline recommendation result includes an offline target recommendation table of at least one user; the offline target recommendation table comprises a plurality of recommendation contents and recommendation weight values of the recommendation contents; the real-time recommendation result comprises a real-time target recommendation table of at least one user; the real-time target recommendation table comprises a plurality of recommendation contents and a recommendation weight value of each recommendation content.

In one possible implementation, the user offline feature data includes one or more of the following: age, gender, user preference information of the user; the user real-time feature data includes one or more of the following: age, gender, user preference information of the user.

In a fourth aspect, the present application provides a recommendation device, the device comprising: a processor and a communication interface; the communication interface is coupled to a processor for running a computer program or instructions to implement the recommendation method as described in any one of the possible implementations of the first aspect and the first aspect.

In a fifth aspect, the present application provides a computer readable storage medium having instructions stored therein which, when run on a terminal, cause the terminal to perform a recommendation method as described in any one of the possible implementations of the first aspect and the first aspect.

In a sixth aspect, embodiments of the present application provide a computer program product comprising instructions which, when run on a recommending apparatus, cause the recommending apparatus to perform the recommending method as described in any of the possible implementations of the first aspect and the first aspect.

In a seventh aspect, embodiments of the present application provide a chip comprising a processor and a communication interface, the communication interface and the processor being coupled, the processor being for running a computer program or instructions to implement the recommendation method as described in any one of the possible implementations of the first aspect and the first aspect.

In the present application, names of the above-mentioned recommended method, system, and storage medium do not constitute limitations on the devices or function modules themselves, and in actual implementation, these devices or function modules may appear under other names. Insofar as the function of each device or function module is similar to the present application, it falls within the scope of the present application and the equivalent technology thereof.

These and other aspects of the application will be more readily apparent from the following description.

The technical scheme provided by the application has at least the following beneficial effects:

based on the technical scheme, when the user is offline, the recommendation system can perform offline calculation according to the Impala algorithm and the Pyspark framework, determine an offline recommendation result, and display the offline recommendation result to the user when the user logs in for the first time; when the user is online, the recommendation system can perform near-real-time calculation to determine the real-time recommendation result, and then the recommendation system displays the real-time recommendation result to the user. Thus, the recommendation system can obtain lower latency and process more data. In addition, based on the offline recommendation result and the real-time recommendation result determined by the technical scheme, the method is closer to the behavior of the user, and can better recommend the user.

Drawings

Fig. 1 is a schematic diagram of a recommendation system according to an embodiment of the present application;

fig. 2 is a schematic hardware structure of a recommendation device according to an embodiment of the present application;

fig. 3 is a schematic hardware structure of another recommending apparatus according to an embodiment of the present application;

FIG. 4 is a flowchart of a recommendation method according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an offline service layer of a recommendation system according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a recommendation system near-real-time service layer and a real-time service layer architecture according to an embodiment of the present application;

FIG. 7 is a schematic diagram of another recommendation system according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a recommendation device according to an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

The character "/" herein generally indicates that the associated object is an "or" relationship. For example, A/B may be understood as A or B.

The terms "first" and "second" in the description and in the claims of the application are used for distinguishing between different objects and not for describing a particular sequential order of objects.

Furthermore, references to the terms "comprising" and "having" and any variations thereof in the description of the present application are intended to cover a non-exclusive inclusion. For example, a process, method, system, article, or apparatus that comprises a list of steps or modules is not limited to only those steps or modules but may include other steps or modules not listed or inherent to such process, method, article, or apparatus.

In addition, in the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "exemplary" or "e.g." should not be taken as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present concepts in a concrete fashion.

In order to facilitate understanding of the technical scheme of the present application, the technical terms related to the present application are described below:

1. recommendation system

At present, the recommendation system is used for recommending the corresponding interface or application to the user according to the behavior preference of the user when the user browses the web interface or uses the target application.

Currently, recommendation systems generally divide architecture into three layers: a real-time service layer (online), a near real-time service layer (online), an offline service layer (offline).

The real-time service layer is mainly used for rapidly responding to the request of the user. In general, since a quick response to a user's request is required, the real-time service layer has a high requirement for time delay, and is required to reach millisecond level. Therefore, the real-time service layer cannot realize larger-scale data processing and model updating, which further results in that the recommendation system is more rigorous in selecting the algorithm model. In order to give consideration to fault tolerance, when the calculation time delay of the real-time service layer exceeds the preset time delay, the calculation needs to be recommended to return a pre-calculated result.

The near real-time service layer is used for completing data processing of a few minutes and a few seconds by using a preset algorithm according to the message queue platform, and can also complete simple model updating.

And the offline service layer is used for storing offline data and running an offline machine learning model. The offline service layer is not limited by the amount of data and the type of algorithm. Accordingly, the offline service layer cannot realize data update and real-time model update with lower time delay.

From the foregoing, it can be seen that real-time services have high requirements on latency in order to quickly respond to user requests and to implement interactions with users. The near real-time service layer cannot update real-time recommendation results in real time. Offline services cannot update data with very low time delay and realize real-time updating of models.

In order to overcome the defects, the application provides a recommendation system, wherein an offline service layer selects a combination of an Impala algorithm and a Pyspark framework, and a near real-time service layer selects a Flink algorithm. Therefore, the recommendation system provided by the application can be used for recommending services to the user with lower time delay while processing more data. In addition, the recommendation system of the application adopts a Redis database with more excellent performance and convenience to store the data of the offline service layer, the near real-time service layer and the real-time service layer in operation.

Embodiments of the present application will be described in detail below with reference to the attached drawings of the specification:

exemplary, as shown in fig. 1, fig. 1 is a recommendation system 100 provided in the present application. The recommendation system 100 specifically includes: an offline service layer 101, a near real-time service layer 102, and a real-time service layer 103.

The offline service layer 101 is configured to store offline data, perform offline computation according to the offline data and the offline feature user model, determine an offline recommendation result, and store the offline recommendation result in a database. Thus, when the user opens the recommendation system, the offline recommendation results are presented to the user with a certain weight. The following describes in detail the functions that the offline service layer 101 can implement:

the offline service layer 101 is configured to determine an offline recommendation result according to the offline feature data of the user and the offline user feature model. The offline user characteristic data are obtained according to an Impala algorithm, and the offline user characteristic model is constructed according to a PySprak framework.

The offline service layer 101 uses an Impala algorithm to obtain a larger improvement of computing performance. This is because the Impala algorithm has a very large improvement in computational performance compared to the algorithms employed in the prior art, and can implement calculations on the hour level when the amount of data is on the Gigabyte (GB) level.

And, the offline service layer 101 uses a PySprak framework to maximize the offline computing power and a more complete learning model. This is because the PySpark framework is an API provided by Spark for Python developers. The machine learning related model of the Python community is very perfect, and the distributed computing capability of Spark is matched, so that the offline computing capability can be furthest exerted.

In a possible implementation manner, the offline service layer 101 is further configured to perform user feature extraction and data preprocessing on offline data according to an Impala algorithm, and determine offline feature data of a user.

It should be noted that the offline data may be pre-stored in the distributed file system (Hadoop Distributed File System, HDFS), and the offline service layer 101 may be capable of acquiring the offline data from the HDFS according to the Impala algorithm.

It will be appreciated that after the user feature extraction, the acquired user offline feature data includes one or more of the following: age, gender, user preference information of the user.

It is understood that the data preprocessing may be data cleansing of offline data. It should be understood that data cleaning is a well-known technical means in the art, and embodiments of the present application are not described in detail herein.

In another possible implementation, the offline service layer 101 is further configured to determine an offline user feature model according to the PySpark framework and the offline user feature data.

Optionally, the offline service layer 101 is further configured to perform task scheduling according to an airlow algorithm. The task scheduling is used for indicating that the offline recommendation result is determined at least one preset time point.

It should be noted that, the task scheduling is performed by adopting the Airflow algorithm, because both the Impala algorithm and the PySpark frame are only responsible for calculation, and the Airflow algorithm is also needed to enable the Impala algorithm and the PySpark frame to run at fixed time, so that the recommendation system can periodically perform offline calculation under the condition that the user is offline, determine an offline recommendation result, and store the offline recommendation result in the database.

It should be noted that, the offline recommendation result is determined according to the offline user feature model, and the offline recommendation result includes at least one offline target recommendation table of the user; the offline target recommendation table comprises a plurality of recommendation contents and a recommendation weight value of each recommendation content.

It will be appreciated that a weight value is a measure that is used to describe or measure the relative importance between objects. The recommendation weight value in the present application refers to a value that decides a recommendation order in a recommendation process, for example: the recommendation weight values of the recommendation A, the recommendation B and the recommendation C are respectively 0.45, 0.20 and 0.35, and the recommendation sequence is the recommendation A, the recommendation C and the recommendation B.

The functions that can be implemented by the offline service layer 101 are described above.

The functions that can be implemented by the near real-time service layer 102 are specifically described below:

the near real-time service layer 102 is configured to process the user behavior log, perform near real-time calculation according to the user behavior log, and determine a real-time recommendation result. Thus, when the user uses the recommendation system for a certain time, the real-time recommendation result is displayed to the user in real time with a certain weight. The functions that can be implemented by the near real-time service layer 102 are specifically described below:

the near real-time service layer 102 is configured to process the user behavior log, and perform near real-time calculation according to the user behavior log to obtain a user feature model, so as to determine a real-time recommendation result according to the user feature model. The user behavior log is stored in Kafka middleware, and the real-time user characteristic model is calculated according to a Flink algorithm.

It will be appreciated that Kafka middleware, written in accordance with the Scala and Java languages, is a high throughput distributed publish-subscribe messaging system that can handle all action flow data of users in a web site.

It should be understood that the near-real-time service selects the Kafka middleware as the storage of the user behavior log, because the Kafka middleware is more convenient to use, and the Kafka middleware can be decoupled, the near-real-time service layer only needs to use data required by near-real-time calculation, and the calculation performance is greatly improved.

And, the near real-time service layer uses the flank algorithm to obtain a larger improvement of the computing performance. The method is characterized in that the algorithm adopted in the prior art processes data in the minute level or the second level, and the Flink algorithm has a great improvement in calculation performance, and can realize millisecond-level updating, so that the calculation performance of a near-real-time service layer can be greatly improved by adopting the Flink algorithm.

In one possible implementation, the near real-time service layer 102 is further configured to determine user real-time feature data according to the flank algorithm in combination with the user behavior log in the Kafka middleware, and update the user real-time feature model accordingly.

Correspondingly, after updating the user real-time feature model, the real-time recommendation result determined according to the updated model is updated. Further, the near real-time service layer 102 stores the updated real-time recommendation result in the Redis database.

The user behavior log refers to behavior data generated when the user accesses the website each time, including a user behavior trace and a traffic log.

It is to be appreciated that decoupling by Kafka middleware can include data splitting of user behavior logs. It should be understood that decoupling is a well-known technical means in the art, and embodiments of the present application are not described in detail herein.

It will be appreciated that, after near real-time computation, the acquired user real-time feature data includes one or more of the following: age, gender, user preference information of the user.

It should be noted that, the real-time recommendation result is determined according to the real-time user feature model, and the real-time recommendation result includes at least one real-time target recommendation table of the user; the real-time target recommendation table comprises a plurality of recommendation contents and a recommendation weight value of each recommendation content. The description of the recommended weight values is referred to above and will not be repeated here.

The functions that can be implemented by the near real-time services layer 102 are described above.

The following describes the functions that can be implemented by the real-time service layer 103 in detail:

the real-time service layer 103 is mainly used for responding to the user request, processing the user request in real time and simulating online service. The real-time service layer 103 also makes recommendations to the user based on the offline recommendation results and the real-time recommendation results. Wherein the offline recommendation result is determined by the offline service layer 101 and the real-time recommendation result is determined by the near real-time service layer 103. Specifically, the real-time service layer 103 is further configured to obtain a user behavior log. The description of the user behavior log is referred to above and will not be repeated here.

The real-time service layer 103 is further configured to obtain a user behavior log through interaction between the user and the recommendation system 100, and store the user behavior log in the Kafka middleware for use by the offline service layer 101 and the near real-time service layer 102.

In addition, the recommendation system 100 further includes: redis database. The Redis database is used for storing data of an offline service layer, a near real-time service layer and a real-time service layer in operation. In addition, the Redis database is also used for storing offline recommendation results and real-time recommendation results.

It should be noted that, the recommendation system 100 adopts the Redis database to store the data of the offline service layer 101, the near real-time service layer 102 and the real-time service layer 103 during operation, which is very convenient in use in consideration of the fact that the Redis database has a simple interface and a performance far superior to other databases.

Based on the above technical solution, the offline service layer 101 selects a combination of the Impala algorithm and the PySpark framework, so that the computing capability can be greatly enhanced; selecting the Redis database to store the data of the offline service layer, the near real-time service layer 102 and the real-time service layer 103 during operation, the performance of the recommendation system 100 can be more superior, and the use is more convenient; the application performs near real-time calculation by the Flink algorithm selected by the near real-time service layer 102, and can realize millisecond-level updating; in addition, the user real-time log is placed in the Kafka middleware, so that the algorithm model can be updated according to the user behavior log by near real-time calculation, and the model is more suitable for the current state of the user. Thus, the recommendation system 100 of the present application enables lower latency and more data to be processed. Further, the recommendation can be better made for the user.

Fig. 2 is a schematic hardware structure diagram of a recommendation device according to an embodiment of the present application. The recommendation device comprises a processor 21, a memory 22, a communication interface 23, a bus 24. The processor 21, the memory 22 and the communication interface 23 may be connected by a bus 24.

The processor 21 is a control center of the recommendation device, and may be one processor or a collective name of a plurality of processing elements. For example, the processor 21 may be a general-purpose central processing unit (central processing unit, CPU), or may be another general-purpose processor. Wherein the general purpose processor may be a microprocessor or any conventional processor or the like.

As one example, processor 21 may include one or more CPUs, such as CPU 0 and CPU 1 shown in fig. 2.

Memory 22 may be, but is not limited to, a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (random access memory, RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), magnetic disk storage or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

In a possible implementation, the memory 22 may exist separately from the processor 21, and the memory 22 may be connected to the processor 21 by a bus 24 for storing instructions or program code. The processor 21, when calling and executing instructions or program code stored in the memory 22, is capable of implementing the resource scheduling method provided in the following embodiments of the present invention.

In another possible implementation, the memory 22 may also be integrated with the processor 21.

The communication interface 23 is used for connecting the recommending device with other devices through a communication network, wherein the communication network can be an ethernet, a wireless access network, a wireless local area network (wireless local area networks, WLAN) and the like. The communication interface 23 may include a receiving unit for receiving data, and a transmitting unit for transmitting data.

Bus 24 may be an industry standard architecture (industry standard architecture, ISA) bus, an external device interconnect (peripheral component interconnect, PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in fig. 2, but not only one bus or one type of bus.

Fig. 3 shows another hardware structure of the recommending apparatus in the embodiment of the present application. As shown in fig. 3, the recommendation device may include a processor 31 and a communication interface 32. The processor 31 is coupled to a communication interface 32.

The function of the processor 31 may be as described above with reference to the processor 21. The processor 31 also has a memory function and can function as the memory 22.

The communication interface 32 is used to provide data to the processor 31. The communication interface 32 may be an internal interface of the recommender or an external interface of the recommender (corresponding to the communication interface 23).

It should be noted that the structure shown in fig. 2 (or fig. 3) does not constitute a limitation of the recommendation device, and the recommendation device may include more or less components than those shown in fig. 2 (or fig. 3), or may combine some components, or may have a different arrangement of components.

Exemplary, as shown in fig. 4, a recommendation method provided in an embodiment of the present application includes the following steps:

s401, the recommendation system responds to login operation of the user and recommends offline recommendation results to the user.

The offline recommendation result is determined according to offline user feature data and an offline user feature model, the offline user feature data is obtained according to an Impala algorithm, and the offline user feature model is constructed according to a PySprak framework.

Alternatively, this step may be performed by the offline service layer 101 in the recommendation system. Specifically, when the user goes offline, the offline service layer 101 extracts offline data from the HDFS according to an Impala algorithm, and performs user feature extraction and data preprocessing on the portion of the offline data through the Impala algorithm to determine offline user feature data; then, the offline service layer 101 performs distributed computation on the offline user feature data according to a Spark algorithm to determine an offline user feature model; finally, the offline service layer 101 learns the offline user feature model according to Python machine learning, determines an offline recommendation result, and stores the offline recommendation result in the Redis database.

Therefore, after the user logs in, the recommendation system only displays the offline recommendation result through the front-end service at the moment because the user behavior log is not acquired yet, and the offline recommendation result is stored in the Redis database.

Note that, when the recommendation switch of the real-time service layer 103 is turned off, the recommendation system 100 only displays the default recommendation content.

And, when the real-time service layer 103 recommendation switch is turned on, the recommendation system 100 first presents the offline recommendation result stored in the dis database, and simultaneously performs the subsequent step S402.

S402, the recommendation system responds to preset operation of a user to determine a user behavior log.

The preset operation of the user may be operations such as sliding, clicking, pressing an entity key, etc., which is not limited in the embodiment of the present application.

Optionally, the real-time services layer 103 generates a user behavior log based on the user's interactions with the front-end services in the recommendation system 100. Accordingly, the real-time services layer 103 stores the user behavior log in the Kafak middleware.

S403, the recommendation system determines real-time recommendation results according to the user behavior log.

The real-time recommendation result is determined according to the real-time user characteristic model and the user real-time characteristic data, and the user real-time characteristic data is determined according to the user behavior log.

Optionally, the near real-time service layer 102 extracts and calculates the user behavior log in the Kafka middleware according to the link algorithm to obtain real-time user feature data, so as to determine a real-time user recommendation model; then, the near real-time service layer 102 determines a real-time recommendation result according to the real-time user recommendation model, and changes the Redis real-time recommendation result, and stores it in the Redis database.

The real-time recommendation result comprises a real-time target recommendation table of at least one user; the real-time target recommendation table comprises a plurality of recommendation contents and a recommendation weight value of each recommendation content. The description of the recommended weight values is referred to above and will not be repeated here.

S404, the recommendation system displays the offline recommendation result and the real-time recommendation result to the user according to the front-end service.

In one possible implementation, the real-time service layer 103 obtains the real-time recommendation result stored in the Redis database through the back-end interface, and displays the real-time recommendation result to the user through the front-end service. The recommendation system 100 displays real-time recommendation results and offline recommendation results to users through front-end services in combination with recommendation weight values. The description of the recommended weight values is referred to above and will not be repeated here.

Exemplary, with reference to fig. 1, as shown in fig. 5, fig. 5 is a schematic diagram of an offline service layer 101 of a recommendation system according to an embodiment of the present application. The following describes the operation procedure of the offline service layer 101 in the recommendation system 100 provided by the present application:

it should be appreciated that the offline service layer 101 recommends portions of the system that are running in the case where the user is offline. The following describes the operation of the offline service layer 101 in connection with fig. 5, in the following steps:

(1) And when the user is offline, the recommendation system performs task scheduling according to an Airflow algorithm.

The Airflow algorithm is used for performing task scheduling and enabling the subsequent steps (2), (3) and (4) to be performed smoothly.

(2) The offline service layer 101 extracts offline data from the HDFS according to the Impala algorithm.

It should be noted that, after extracting the offline data from the HDFS according to the Impala algorithm, the offline service layer 101 performs user feature extraction and data preprocessing according to the portion of the offline data by the Impala algorithm, and determines the offline user feature data.

(3) The offline service layer 101 performs distributed computation on the offline user feature data obtained in the step (2) according to the Spark algorithm, and determines an offline user feature model.

(4) The offline service layer 101 learns the offline user feature model obtained in the step (3) according to Python machine learning, and determines an offline recommendation result.

The offline recommendation result comprises an offline target recommendation table of at least one user; the offline target recommendation table comprises a plurality of recommendation contents and a recommendation weight value of each recommendation content. The description of the recommended weight values is referred to above and will not be repeated here.

(5) The offline service layer 101 stores the offline recommendation result determined in step (4) according to the Redis database.

The above describes the workflow of the offline service layer 101 in the recommendation system 100.

As shown in fig. 6 with reference to fig. 1, fig. 6 is a schematic architecture diagram of a near-real-time service layer 102 and a real-time service layer 103 of a recommendation system according to an embodiment of the present application, and the following describes the operation process of the near-real-time service layer 102 and the real-time service layer 103 in the recommendation system 100 according to the present application:

the near real-time service layer 102 and the real-time service layer 103 are parts that operate in the recommendation system when the user is online. Accordingly, the combined description is made herein in view of the close correlation between the near real-time service layer 102 and the real-time service layer 103.

The following describes the operation of the near real-time service layer 102 and the real-time service layer 103 in steps with reference to fig. 6:

(1) The recommendation system 100 only presents default recommended content when the recommendation switch of the real-time services layer 103 is closed.

And, when the real-time service layer 103 recommends that the switch is opened, the recommendation system 100 performs the subsequent step (2).

(2) The real-time services layer 103 generates user behavior logs based on user interactions with front-end services in the recommendation system 100. Accordingly, the real-time services layer 103 stores the user behavior log in the Kafak middleware.

(3) The near real-time service layer 102 extracts and calculates the user behavior log in the Kafka middleware according to the link algorithm to obtain real-time user characteristic data, so as to determine a real-time user recommendation model.

(4) The near real-time service layer 102 determines real-time recommendation results according to the real-time user recommendation model, and changes the Redis real-time recommendation results, and stores the Redis real-time recommendation results in the Redis database.

(5) The real-time service layer 103 obtains real-time recommendation results stored in the Redis database through a back-end interface, and displays the real-time recommendation results to the user through the front-end service. The recommendation system 100 presents real-time recommendation results to users through front-end services in combination with recommendation weight values. The description of the recommended weight values is referred to above, and will not be repeated here.

The above describes the workflow of the near real-time services layer 102 and the real-time services layer 103 in the recommendation system 100.

In one possible implementation, the offline service layer 101, the near real-time service layer 102, and the real-time service layer 103 may be combined to operate together to achieve better recommendation.

As shown in fig. 7, fig. 7 is a schematic diagram of an architecture of another recommendation system according to an embodiment of the present application, and the following steps describe a workflow of the recommendation system 100 according to the present application:

And, when the real-time service layer 103 recommends that the switch is turned on, the recommendation system 100 first presents the offline recommendation result stored in the dis database and simultaneously performs the subsequent step (2).

(5) The real-time service layer 103 obtains real-time recommendation results stored in the Redis database through a back-end interface, and displays the real-time recommendation results to the user through the front-end service. The recommendation system 100 presents real-time recommendation results to users through front-end services in combination with recommendation weight values. The description of the recommended weight values is referred to above and will not be repeated here.

Optionally, the present embodiment further includes step (6) after step (5).

And (6) when the user is in an offline state, executing the steps in the embodiment corresponding to fig. 4, so that an offline recommendation result is obtained and stored in the Redis database.

Based on the above technical scheme, three layers of the offline service layer 101, the near real-time service layer 102 and the real-time service layer 103 are combined together to meet various recommendation scenes and various business requirements, and the offline recommendation and the real-time update of the recommendation when the user is online can be realized, so that the user has better experience.

Exemplary, as shown in fig. 8, a schematic structural diagram of a recommendation device according to an embodiment of the present application is provided. The recommending means is for executing the recommending method shown in fig. 4. The recommending means may comprise a processing unit 201 and a storage unit 202.

A processing unit 201, configured to recommend an offline recommendation result to a user in response to a login operation of the user; the offline recommendation result is determined according to offline user feature data and an offline user feature model, wherein the offline user feature data is obtained according to an Impala algorithm, and the offline user feature model is constructed according to a PySprak framework;

in one possible implementation, the processing unit is further configured to determine a user behavior log in response to a preset operation of the user;

in one possible implementation, the processing unit is further configured to determine a real-time recommendation result according to the user behavior log; the real-time recommendation result is determined according to the real-time user characteristic model and the user real-time characteristic data, and the user real-time characteristic data is determined according to the user behavior log;

In a possible implementation manner, the processing unit is further configured to update the real-time user feature model according to a link algorithm.

The storage unit 202 is configured to store the offline recommendation result and the real-time recommendation result according to the Redis database.

In a possible implementation, the storage unit 202 is further configured to store the user behavior log in Kafka middleware.

In addition, the technical effects of the recommending apparatus described in fig. 8 may refer to the technical effects of the recommending method described in the foregoing embodiments, and will not be described herein.

From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be implemented by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to implement all or part of the functions described above. The specific working processes of the above-described systems, devices and units may refer to the corresponding processes in the foregoing method embodiments, which are not described herein.

An embodiment of the application provides a computer program product comprising instructions which, when run on a computer, cause the computer to perform the recommended method of the method embodiment described above.

The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores instructions which, when run on a computer, cause the computer to execute the recommended method in the method flow shown in the method embodiment.

The computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access Memory (Random Access Memory, RAM), a Read-Only Memory (ROM), an erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), a register, a hard disk, an optical fiber, a portable compact disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing, or any other form of computer readable storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an application specific integrated circuit (Application Specific Integrated Circuit, ASIC). In embodiments of the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

Since the recommending apparatus, the computer readable storage medium, and the computer program product in the embodiments of the present application can be applied to the above-mentioned method, the technical effects that can be obtained by the recommending apparatus, the computer readable storage medium, and the computer program product can also refer to the above-mentioned method embodiments, and the embodiments of the present application are not described herein again.

The foregoing is merely a specific embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present application should be covered by the scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A recommendation method, the method comprising:

recommending an offline recommendation result to the user in response to a login operation of the user; the offline recommendation result is determined according to user offline feature data and an offline user feature model, wherein the user offline feature data is obtained according to an Impala algorithm, and the offline user feature model is constructed according to a PySprak framework;

responding to a preset operation of a user, and determining a user behavior log;

determining a real-time recommendation result according to the user behavior log; the real-time recommendation result is determined according to a real-time user characteristic model and user real-time characteristic data, and the user real-time characteristic data is determined according to the user behavior log;

And displaying the offline recommendation result and the real-time recommendation result to a user according to the front-end service.

2. The method according to claim 1, wherein the method further comprises:

and updating the real-time user characteristic model according to a Flink algorithm.

3. The method of claim 2, wherein the offline recommendation result and the real-time recommendation result are stored in a Redis database.

4. A method according to claim 3, wherein the user behaviour log is stored in Kafka middleware.

5. A recommendation device, characterized in that the recommendation device comprises: a processing unit;

the processing unit is used for responding to the login operation of the user and recommending an offline recommendation result to the user; the offline recommendation result is determined according to user offline feature data and an offline user feature model, wherein the user offline feature data is obtained according to an Impala algorithm, and the offline user feature model is constructed according to a PySprak framework;

the processing unit is further used for responding to preset operation of a user and determining a user behavior log;

the processing unit is further used for determining a real-time recommendation result according to the user behavior log; the real-time recommendation result is determined according to a real-time user characteristic model and user real-time characteristic data, and the user real-time characteristic data is determined according to the user behavior log;

The processing unit is further configured to display the offline recommendation result and the real-time recommendation result to a user according to the front-end service.

6. The recommendation device of claim 5, wherein the processing unit is further configured to update the real-time user feature model according to the link algorithm.

7. The recommendation device of claim 6, further comprising: a storage unit;

the storage unit is used for storing the offline recommendation result and the real-time recommendation result in the Redis database.

8. The recommendation device of claim 7, wherein the storage unit is further configured to store the user behavior log in the Kafka middleware.

9. A recommendation system, characterized in that it is applied to the recommendation method of claims 1 to 4, comprising: an offline service layer, a near real-time service layer and a real-time service layer;

the offline service layer is used for determining an offline recommendation result according to the offline characteristic data of the user and the offline user characteristic model; the user offline characteristic data are obtained according to an Impala algorithm, and the offline user characteristic model is constructed according to a PySprak framework;

The near real-time service layer is used for determining real-time recommendation results according to the real-time characteristic data of the user;

the real-time service layer is used for responding to the user request and acquiring a user behavior log; the user behavior log is used for determining the user real-time characteristic data.

10. The recommendation system according to claim 9, wherein said offline service layer is further configured to obtain offline data from a distributed file system HDFS according to said Impala algorithm;

the offline service layer is further used for carrying out user characteristic extraction and data preprocessing on the offline data according to the Impala algorithm, and determining the offline characteristic data of the user;

the offline service layer is further configured to determine the offline user feature model according to the PySpark framework and user offline feature data.

11. The recommendation system of claim 10, wherein said near real-time service layer is further configured to update a real-time user feature model according to a link algorithm; the real-time user feature model is used for determining the real-time recommendation result by the recommendation system.

12. The recommendation system of claim 11, further comprising: a Redis database;

The Redis database is used for storing the data of the offline service layer, the near real-time service layer and the real-time service layer in operation;

the Redis database is also used for storing the offline recommendation result and the real-time recommendation result.

13. The recommendation system of claim 12, wherein the real-time service layer comprises a Kafka middleware;

the Kafka middleware is used for storing the user behavior log.

14. The recommendation system of claim 13, wherein said offline service layer is further configured to schedule tasks according to an airlow algorithm; the task schedule is used for indicating that the offline recommendation result is determined at least at one preset time point.

15. The recommendation system of claim 14, wherein said offline recommendation results comprise an offline target recommendation table for at least one user; the offline target recommendation table comprises a plurality of recommendation contents and a recommendation weight value of each recommendation content;

the real-time recommendation result comprises a real-time target recommendation table of at least one user; the real-time target recommendation table comprises a plurality of recommendation contents and a recommendation weight value of each recommendation content.

16. The recommendation system according to any one of claims 9 to 15, wherein said user offline feature data comprises one or more of: age, gender, user preference information of the user;

the user real-time feature data includes one or more of the following: age, gender, user preference information of the user.

17. A recommendation device, comprising: a processor and a communication interface; the communication interface is coupled to the processor for running a computer program or instructions to implement the recommendation method as claimed in any of the claims 1-4.

18. A computer readable storage medium having instructions stored therein, characterized in that, when the instructions are executed by a computer, the computer performs the recommendation method as claimed in any one of the preceding claims 1-4.