CN107566450B - Data processing method and device for real-time user behavior and electronic equipment - Google Patents
Data processing method and device for real-time user behavior and electronic equipment Download PDFInfo
- Publication number
- CN107566450B CN107566450B CN201710650747.5A CN201710650747A CN107566450B CN 107566450 B CN107566450 B CN 107566450B CN 201710650747 A CN201710650747 A CN 201710650747A CN 107566450 B CN107566450 B CN 107566450B
- Authority
- CN
- China
- Prior art keywords
- data
- filtering
- filtered
- category
- streaming
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The embodiment of the invention provides a data processing method, a device and electronic equipment for real-time user behavior, wherein the method comprises the following steps: acquiring data of real-time user behaviors; shunting data according to a preset shunting rule to obtain first shunting data and second shunting data; filtering the first streaming data through a first data filtering engine corresponding to the first streaming data to obtain a first filtered data set, and filtering the second streaming data through a second data filtering engine corresponding to the second streaming data to obtain a second filtered data set; and responding all data in the first filtered data set and all data in the second filtered data set, determining all the first messages to be pushed and all the second messages to be pushed, and pushing all the first messages to be pushed and all the second messages to be pushed to corresponding users. By applying the embodiment of the invention, the real-time user behavior can be fed back in time, so that the user experience is improved.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data processing method and apparatus for real-time user behavior, and an electronic device.
Background
Data processing based on real-time user behavior is a technology which is gradually popular in recent years, and is widely applied to various products of large-scale internet companies. The product experience of the user is adjusted and fed back in real time through the real-time behaviors of the user collected by the background, so that the product can be optimized, and information can be pertinently recommended to the user. Typical products based on real-time user behaviors include real-time recommendations, real-time information push, and the like.
As shown in fig. 1, an existing data processing method based on real-time user behaviors delivers various behaviors of a real-time user separately, and stores various behaviors of the real-time user in a collection background corresponding to the user behavior, such as a user behavior 1 collection background, a user behavior 2 collection background … … user behavior y collection background, and the like.
However, the inventor finds that the prior art has at least the following problems in the process of implementing the invention: in the existing data Processing method for user behaviors, although relevant messages can be quickly pushed to different users by performing response Processing on data of real-time user behaviors, when the data volume of a certain real-time user behavior acquisition background is large, for example, when the real-time user behavior is a video playing operation, because there are many users performing the playing operation at the same time, if the data of the playing operation are processed at the same time, the consumption of a Central Processing Unit (CPU) is too high, and the Processing process consumes a long time, so that the real-time user behaviors cannot be fed back in time, and the user experience is reduced.
Disclosure of Invention
The embodiment of the invention aims to provide a data processing method and device for real-time user behaviors and electronic equipment, so that the real-time user behaviors can be fed back in time, and the user experience is improved. The specific technical scheme is as follows:
the embodiment of the invention discloses a data processing method for real-time user behavior, which comprises the following steps:
acquiring data of real-time user behaviors;
shunting the data according to a preset shunting rule to obtain first shunting data and second shunting data;
filtering the first streaming data through a first data filtering engine corresponding to the first streaming data to obtain a first filtered data set containing first data of each category, and filtering the second streaming data through a second data filtering engine corresponding to the second streaming data to obtain a second filtered data set containing second data of each category;
responding to all data in the first filtered data set and all data in the second filtered data set, determining each first message to be pushed corresponding to each category of first data and each second message to be pushed corresponding to each category of second data, and pushing each first message to be pushed and each second message to be pushed to corresponding users.
Optionally, after the first filtering data set including the first data of each category is obtained by filtering the first streaming data through the first data filtering engine corresponding to the first streaming data, and the second filtering data set including the second data of each category is obtained by filtering the second streaming data through the second data filtering engine corresponding to the second streaming data, the method further includes:
storing all data in the first filtered data set and all data in the second filtered data set in a message queue system;
correspondingly, responding to all data in the first filtered data set and all data in the second filtered data set, determining each first message to be pushed corresponding to each category of first data and each second message to be pushed corresponding to each category of second data, and pushing each first message to be pushed and each second message to be pushed to corresponding users, including:
and acquiring all data in the first filtered data set and all data in the second filtered data set from the message queue system, responding to all data in the first filtered data set and all data in the second filtered data set, determining each first message to be pushed corresponding to each category of first data and each second message to be pushed corresponding to each category of second data, and pushing each first message to be pushed and each second message to be pushed to corresponding users.
Optionally, the shunting the data according to a preset shunting rule to obtain first shunting data and second shunting data includes:
acquiring the priority, delay and flow of each data in the data;
shunting data of which the priority is higher than a preset priority and the delay is lower than a preset delay in the data to obtain first shunting data;
and shunting the data with the flow larger than a preset threshold value in the data to obtain second shunting data.
Optionally, the filtering, by the first data filtering engine corresponding to the first streaming data, the first streaming data to obtain a first filtered data set including first data of each category, and filtering, by the second data filtering engine corresponding to the second streaming data, the second streaming data to obtain a second filtered data set including second data of each category, includes:
acquiring a first filtering logic set containing all filtering logic in the first data filtering engine and a second filtering logic set containing all filtering logic in the second data filtering engine;
sequentially filtering the first streaming data through each filtering logic in the first filtering logic set to obtain the first filtering data set containing first data of each category;
and sequentially filtering the second sub-stream data through each filtering logic in the second filtering logic set to obtain a second filtering data set containing second data of each category.
Optionally, the sequentially filtering, by each filtering logic in the first filtering logic set, the first streaming data to obtain the first filtered data set including first data of each category, includes:
sending the first streaming data to a different thread group;
and respectively filtering data included in different thread groups through the filtering logic corresponding to each thread group in the first filtering logic set to obtain the first filtering data set including the filtering data corresponding to each thread group.
Optionally, after the data is shunted according to a preset shunting rule to obtain first shunting data and second shunting data, the method further includes:
storing the first and second streaming data in a Nginx log;
correspondingly, the filtering the first streaming data by the first data filtering engine corresponding to the first streaming data to obtain a first filtered data set including first data of each category, and filtering the second streaming data by the second data filtering engine corresponding to the second streaming data to obtain a second filtered data set including second data of each category, includes:
and filtering the first streaming data in the Nginx log through a first data filtering engine corresponding to the first streaming data to obtain a first filtered data set containing first data of each category, and filtering the second streaming data in the Nginx log through a second data filtering engine corresponding to the second streaming data to obtain a second filtered data set containing second data of each category.
The embodiment of the invention also discloses a data processing device for real-time user behavior, which comprises:
the acquisition module is used for acquiring data of real-time user behaviors;
the shunting module is used for shunting the data according to a preset shunting rule to obtain first shunting data and second shunting data;
the filtering module is used for filtering the first streaming data through a first data filtering engine corresponding to the first streaming data to obtain a first filtered data set containing first data of each category, and filtering the second streaming data through a second data filtering engine corresponding to the second streaming data to obtain a second filtered data set containing second data of each category;
and the processing module is used for responding to all data in the first filtered data set and all data in the second filtered data set, determining each first message to be pushed corresponding to each category of first data and each second message to be pushed corresponding to each category of second data, and pushing each first message to be pushed and each second message to be pushed to corresponding users.
Optionally, the apparatus further comprises:
a first storage module, configured to store all data in the first filtered data set and all data in the second filtered data set in a message queue system;
correspondingly, the processing module is specifically configured to obtain all data in the first filtered data set and all data in the second filtered data set from the message queue system, respond to all data in the first filtered data set and all data in the second filtered data set, determine each first message to be pushed corresponding to each category of first data and each second message to be pushed corresponding to each category of second data, and push each first message to be pushed and each second message to be pushed to a corresponding user.
Optionally, the shunting module includes:
the first obtaining submodule is used for obtaining the priority, delay and flow of each data in the data;
the first streaming submodule is used for streaming data of which the priority is higher than a preset priority and the delay is lower than a preset delay in the data to obtain first streaming data containing first data of each category;
and the second shunting submodule is used for shunting the data of which the flow is greater than a preset threshold value in the data to obtain second shunting data containing second data of each category.
Optionally, the filtering module includes:
a second obtaining sub-module, configured to obtain a first filtering logic set including all filtering logic in the first data filtering engine and a second filtering logic set including all filtering logic in the second data filtering engine;
the first filtering submodule is used for sequentially filtering the first streaming data through each filtering logic in the first filtering logic set to obtain the first filtering data set containing first data of each category;
and the second filtering submodule is used for sequentially filtering the second sub-stream data through each filtering logic in the second filtering logic set to obtain the second filtering data set containing the second data of each category.
Optionally, the first filtering sub-module includes:
a sending unit, configured to send the first streaming data to different thread groups;
and the filtering unit is used for filtering data included in different thread groups respectively through the filtering logics of the thread groups corresponding to the first filtering logic set to obtain the first filtering data set containing the filtering data corresponding to each thread group.
Optionally, the apparatus further comprises:
a second storage module to store the first and second streaming data in a Nginx log;
correspondingly, the filtering module is specifically configured to filter the first streaming data in the Nginx log by using a first data filtering engine corresponding to the first streaming data to obtain a first filtered data set including first data of each category, and filter the second streaming data in the Nginx log by using a second data filtering engine corresponding to the second streaming data to obtain a second filtered data set including second data of each category.
The embodiment of the invention also discloses electronic equipment which comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory finish mutual communication through the communication bus;
the memory is used for storing a computer program;
the processor is used for realizing the steps of the data processing method of the real-time user behavior when executing the program stored in the memory.
In yet another aspect of the present invention, a computer-readable storage medium is further disclosed, wherein instructions are stored in the computer-readable storage medium, and when the instructions are executed on a computer, the computer is caused to execute any one of the above-mentioned data processing methods for real-time user behavior.
In another aspect of the present invention, an embodiment of the present invention further discloses a computer program product containing instructions, which when run on a computer, causes the computer to execute any one of the above-mentioned data processing methods for real-time user behavior.
The embodiment of the invention provides a data processing method, a device and electronic equipment for real-time user behaviors, which comprises the steps of firstly acquiring data of the real-time user behaviors; then shunting the data according to a preset shunting rule to obtain first shunting data and second shunting data; then, filtering the first streaming data through a first data filtering engine corresponding to the first streaming data to obtain a first filtered data set containing first data of each category, and filtering the second streaming data through a second data filtering engine corresponding to the second streaming data to obtain a second filtered data set containing second data of each category; and finally, responding to all data in the first filtered data set and all data in the second filtered data set, determining each first message to be pushed corresponding to each category of first data and each second message to be pushed corresponding to each category of second data, and pushing each first message to be pushed and each second message to be pushed to corresponding users. The data of all real-time user behaviors are shunted firstly, and then the shunted data of the real-time user behaviors are filtered respectively, so that the flexibility of real-time recommendation system change is met, the real-time user behaviors are fed back quickly, and the user experience is improved. Of course, it is not necessary for any product or method of practicing the invention to achieve all of the above-described advantages at the same time.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a diagram illustrating a data processing method for real-time user behavior in the prior art;
fig. 2 is a schematic flowchart of a data processing method for real-time user behavior according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a data processing method for real-time user behavior according to an embodiment of the present invention;
fig. 4 is a system architecture diagram of a data processing method for real-time user behavior according to an embodiment of the present invention;
fig. 5 is another schematic flow chart of a data processing method for real-time user behavior according to an embodiment of the present invention;
FIG. 6 is a block diagram of a data filtering method according to an embodiment of the present invention;
FIG. 7 is a block diagram of another data filtering method according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of a data processing apparatus for real-time user behavior according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention.
In recent years, data processing methods for real-time user behaviors have been widely applied to various products of large-scale internet companies, and especially for companies that rely on real-time recommendation and real-time feedback products, such as news information APPs, short video APPs, e-commerce APPs, and the like, data processing methods for real-time user behaviors are very important. However, in the existing data processing method for real-time user behaviors, different real-time user behaviors are collected in corresponding backgrounds, and then data of each background is processed, so that the flexibility of requirement change on products cannot be met.
Therefore, the invention provides a data processing method of real-time user behaviors, which comprises the steps of shunting all data of the real-time user behaviors, dividing the data into two types of data, respectively filtering the shunted data of the real-time user behaviors, namely respectively filtering the shunted data of the two types of data to obtain filtered subdivided data, and respectively processing (calculating, analyzing, counting and the like) the subdivided data, so that different processing can be carried out aiming at data requirements of different data magnitude, different data delay, different data response and the like, the flexibility of real-time recommendation system change is met, the quick feedback of the real-time user behaviors is realized, and the user experience is improved. The specific implementation method comprises the following steps:
referring to fig. 2, fig. 2 is a schematic flow chart of a data processing method for real-time user behavior according to an embodiment of the present invention, including the following steps:
s201, acquiring data of real-time user behaviors.
Specifically, in the real-time recommendation system, the system may recommend information related to the user according to data from the real-time user behavior, and the recommendation is based on the premise that the data of the real-time user behavior is obtained by the system. Here, the data of the real-time user behavior refers to an operation behavior of the user at the current time in the real-time recommendation system, for example, a browsing operation in which the user clicks a certain piece of news in an application of news information, an operation in which the user watches a certain type of video in an application of short video, and the like.
S202, shunting the data according to a preset shunting rule to obtain first shunting data and second shunting data.
In particular, for a real-time recommendation system, the data of real-time user behaviors in the system is very large, and the data of various real-time user behaviors have different characteristics. For example, the priority of the data is different, the delay of the data is different, the traffic of the data is different, the format of the data is different, and so on.
In the embodiment of the invention, in order to meet the flexible requirement change of various real-time products, various behaviors of a user can be uniformly collected, namely the acquired data of the real-time user behaviors are finely split according to the requirements of the products in the background. Therefore, the method and the device can firstly shunt the acquired data of all the real-time user behaviors, namely classify the data of all the real-time user behaviors, and can carry out different processing on different data, thereby meeting the data requirements and real-time requirements of different products.
S203, filter the first streaming data through the first data filtering engine corresponding to the first streaming data to obtain a first filtered data set including first data of each category, and filter the second streaming data through the second data filtering engine corresponding to the second streaming data to obtain a second filtered data set including second data of each category.
Specifically, after all real-time user behavior data are shunted, two types of data are obtained, namely first shunt data and second shunt data, and the first shunt data and the second shunt data contain various user behavior data, so that the first shunt data and the second shunt data need to be filtered, and the filtered fine data can be obtained.
Here, a specific filtering manner may be to filter the first partial data through the first data filtering engine to obtain a set of all the filtered partial data, that is, a first filtered data set including the first data of each category. For example, the first data of each category included in the first filtered data set may be data of a hotspot video browsed by a user at the same time, data recommended by the user in real time when the user clicks, and the like.
Similarly, the second data filtering engine filters the second sub-stream data to obtain a set of all the filtered sub-stream data, that is, a second filtered data set including the second data of each category. For example, the second data including each category included in the second filtered data set may be data for searching for the same popular drama at the same time, data for selecting the same variety program, data for clicking to play the same video at the same time, data for registering an account at the same time, or the like.
The above-mentioned first data of each category and the second data of each category are mainly used for distinguishing different categories of data in the first filtered data set and the second filtered data set. The first data filtering engine can only filter the first shunt data, and the second data filtering engine can only filter the second shunt data. The first data filtering engine and the second data filtering engine may be filtering engines that process different data, for example, the first data filtering engine may be a high speed filtering engine to process all high priority, low latency data, the second data filtering engine may be a high performance filtering engine to process all high traffic data, etc.
And S204, responding to all data in the first filtered data set and all data in the second filtered data set, determining each first message to be pushed corresponding to each category of first data and each second message to be pushed corresponding to each category of second data, and pushing each first message to be pushed and each second message to be pushed to corresponding users.
Specifically, the data is filtered to obtain data of different feature types, the data of different feature types are respectively subjected to response processing, each first message to be pushed corresponding to each category of first data and each second message to be pushed corresponding to each category of second data are determined, and each first message to be pushed and each second message to be pushed are pushed to corresponding users. For example, all users involved in acquiring any category of first data, that is, source users of the category of first data, may be determined, and then the first to-be-pushed message corresponding to the category of first data may be pushed to all the determined users. Therefore, different data are processed differently, the processing speed of the data of the real-time user behavior can be improved, and the related information can be pushed to the user more quickly.
Therefore, according to the data processing method for the real-time user behavior provided by the embodiment of the invention, the data of the real-time user behavior is obtained; then shunting the data according to a preset shunting rule to obtain first shunting data and second shunting data; filtering the first streaming data by a first data filtering engine corresponding to the first streaming data to obtain a first filtered data set containing first data of each category, and filtering the second streaming data by a second data filtering engine corresponding to the second streaming data to obtain a second filtered data set containing second data of each category; and finally, responding all data in the first filtered data set and all data in the second filtered data set, determining all first messages to be pushed corresponding to all kinds of first data and all second messages to be pushed corresponding to all kinds of second data, and pushing all the first messages to be pushed and all the second messages to be pushed to corresponding users. The data of all real-time user behaviors are shunted firstly, and then the shunted data of the real-time user behaviors are filtered respectively, so that the flexibility of real-time recommendation system change is met, the real-time user behaviors are fed back quickly, and the user experience is improved.
As shown in fig. 3, in the real-time recommendation system, all data for uniformly delivering user behaviors are collected by a real-time user behavior data collection background, and then the real-time user behavior data is split according to requirements to obtain various required user behaviors, such as user behavior 1, user behavior 2, user behavior 3 … …, user behavior x, user behavior y, and the like. And finally, the user behaviors are respectively processed, so that the information related to the user behaviors is recommended to the user.
In an optional embodiment of the present invention, after obtaining the first filtered data set including the first data of each category and the second filtered data set including the second data of each category, all data in the first filtered data set and all data in the second filtered data set may be further stored in the message queue system.
In particular, message queuing is Microsoft's message processing technology that provides message processing and message queuing functions for any application in any combination of Microsoft windows-equipped computers, whether or not the computers are on the same network or are online at the same time. It can also be said that a message queue system is a container that holds messages during their transmission. Where the primary purpose of the queue is to provide routing and to ensure delivery of the message, if the recipient is unavailable at the time the message is sent, the message queue will hold the message until it can be successfully delivered. All data in the first filtering data set and all data in the second filtering data set are stored in the message queue system, so that the rapid and accurate transmission of messages is guaranteed, and meanwhile, a back-end statistics team or a calculation team obtains data of real-time user behaviors from the message queue system for calculation processing.
Correspondingly, all data in the first filtered data set and all data in the second filtered data set are responded, each first message to be pushed corresponding to each category of first data is determined, and each second message to be pushed corresponding to each category of second data, and when each first message to be pushed and each second message to be pushed are pushed to the corresponding user, all data in the first filtered data set and all data in the second filtered data set may be retrieved from the message queue system, responding all data in the first filtered data set and all data in the second filtered data set, determining each first message to be pushed corresponding to each category of first data, and pushing the first messages to be pushed and the second messages to be pushed to corresponding users.
Specifically, all data in the first filtered data set and all data in the second filtered data set are stored in the message queue system, so that the data are directly called from the message queue system when being processed, the condition that the data are not lost in the data transmission process is ensured, and the accuracy of data processing is improved.
In an optional embodiment of the present invention, when data is shunted according to a preset shunting rule to obtain first shunting data and second shunting data, the method specifically includes:
first, the priority, delay and flow of each data in the data are obtained.
Specifically, the data splitting is realized according to a preset splitting rule. In this embodiment, the preset shunting rule may be: the data is split according to the priority information of each data, the delay information of the data and the flow information of the processing data, and the data is divided into two types, namely first split data and second split data. It should be noted that, here, the data is shunted through three information, that is, the priority, the delay, and the traffic of the data, which is only an optional embodiment provided by the present invention, and all methods that satisfy the preset shunting rule belong to the protection scope of the present invention.
And secondly, shunting the data of which the priority is higher than the preset priority and the delay is lower than the preset delay to obtain first shunting data.
Specifically, the data with the priority higher than the preset priority and the delay lower than the preset delay in all the acquired data of the real-time user behavior is shunted into the first shunt data. Here, the size of the preset priority and the size of the preset delay may be set by actual conditions.
And thirdly, shunting the data with the flow larger than a preset threshold value in the data to obtain second shunting data.
Specifically, the data of the data with the flow larger than the preset threshold in all the acquired data of the real-time user behavior is shunted into the second shunt data. Similarly, the magnitude of the preset threshold is also set by actual conditions. For example, if there are many users performing the same video playing operation at the same time, the flow rate of the data of the video playing operation is also very large, and if the flow rate of the data is greater than the preset threshold, the data of the video playing operation is divided into the second streaming data.
In this embodiment of the present invention, filtering first bitstream data by a first data filtering engine corresponding to the first bitstream data to obtain a first filtered data set including first data of each category, and filtering second bitstream data by a second data filtering engine corresponding to the second bitstream data to obtain a second filtered data set including second data of each category, includes:
the first step is to obtain a first filtering logic set containing all filtering logic in the first data filtering engine and a second filtering logic set containing all filtering logic in the second data filtering engine.
Specifically, the filtering of the data is performed through a corresponding filtering logic, for example, for a data related to video playing, if the preset filtering logic is: all the data of the video containing the keywords of the animation can be filtered out according to the filtering logic. The filtering logic in the invention can be added and modified in the background, because different real-time recommendation systems have different contents to be pushed to users, and the same real-time recommendation system has different recommended contents in different periods, such as a video recommendation system, a video played in the last month is A, and the video played in the next month may be changed into B, and the like. Therefore, each data filtering engine can have a plurality of different filtering logics so as to meet the flexible requirement change of various real-time recommendation systems.
Step two, sequentially filtering the first streaming data through each filtering logic in the first filtering logic set to obtain a first filtering data set containing first data of each category; and sequentially filtering the second sub-stream data through each filtering logic in the second filtering logic set to obtain a second filtering data set containing second data of each category.
Specifically, the first filtering logic set and the second filtering logic set both include a plurality of filtering logics, and the first filtering data set including the first data of each category is obtained by filtering the first partial flow data through each filtering logic in the first filtering logic set according to the filtering logics in sequence. Similarly, the second filtering data set containing the second data of each category is obtained by sequentially filtering the second sub-stream data through each filtering logic in the second filtering logic set. Here, the filtered data obtained by different filtering logics are different, and since the filtering logics are set according to the requirements of the product, the filtered data also meets the flexible requirement change of the product.
Referring to fig. 4, fig. 4 is a system architecture diagram of a data processing method for real-time user behavior according to an embodiment of the present invention, where the system architecture diagram includes: a data receiving module 401 of real-time user behavior, a high-speed filtering engine 402, a high-performance filtering engine 403, a background management system 404, and a message queue system 405.
Generally, the real-time user behavior is sent through a post HTTP request, and the data receiving module 401 of the real-time user behavior in the real-time recommendation system receives the data of the real-time user behavior in the request. The data receiving module 401 of user behavior transmits the received data of real-time user behavior, i.e., raw data, to the high-speed filtering engine 402 and the high-performance filtering engine 403.
Here, the high-speed filtering engine 402 is used as one of the first data filtering engine and the second data filtering engine proposed by the present invention, and the first data filtering engine and the second data filtering engine are named in the present invention to distinguish two different data filtering engines, and in this embodiment, one of them may be used as the high-speed filtering engine 402, and the other one may be used as the high-performance filtering engine 403. Since the high-speed filtering engine 402 can only filter all data that conforms to the high-speed filtering engine 402, the received data of the real-time user behavior is shunted to satisfy the data to be filtered by the high-speed filtering engine 402. The high-speed filtering engine 402 is a high-speed real-time data filtering engine based on regular expression and character string text matching, and rapidly filters data meeting requirements through filtering logic configured by a user. Here, high speed filter engine 402 processes all high priority, low latency data.
Like the high-speed filtering engine 402, the high-performance filtering engine 403 is used as one of the first data filtering engine and the second data filtering engine proposed by the present invention, and since the high-performance filtering engine 403 can only filter all data that conforms to the high-performance filtering engine 403, the received data of the real-time user behavior is split into data that satisfies the data to be filtered by the high-performance filtering engine 403. The high-performance filtering engine 403 is a filtering engine based on a big data real-time computing system, and implements filtering and splitting of the super-large flow data according to a specified rule through a distributed computing framework, and supports horizontal expansion of quantity processing capability. Here, the high performance filtering engine 403 handles all data with large traffic. For example, when one of the filtering logics of the high performance filtering engine 403 is the data of the barrage related to a certain video, and the filtered data is related to the barrage of the video, if the product manager needs to view the report of the barrage data of the video, the product manager can directly view the filtered data from the high performance filtering engine.
Specifically, to filter data through the high-speed filtering engine 402 and the high-performance filtering engine 403, the data filtering logic may be obtained from the background management system 404. The data filtering logic in the background management system 404 may also be added and modified according to actual needs.
After the data is filtered by the high-speed filtering engine 402 and the high-performance filtering engine 403, the filtered fine data is obtained, and the fine data is transmitted to the message queue system 405, so that the rapid and accurate transmission of the message is ensured, and meanwhile, a back-end statistics team, an analysis team or a calculation team obtains the data of the real-time user behavior from the message queue system to perform the processing of calculation, analysis, statistics and the like.
An embodiment of the present invention further provides a specific process of the processing method, and referring to fig. 5, fig. 5 is another schematic flow diagram of the data processing method for real-time user behavior provided in the embodiment of the present invention, and the process includes the following steps:
firstly, data of user operation behaviors are delivered to Nginx, then all data in the Nginx are shunted, the shunting process is that the data acquisition device 1 acquires data with high priority and low delay in the Nginx, and the data acquisition device 2 acquires data with large flow in the Nginx, so that the data are shunted successfully. And then, filtering the data in the data acquisition device 1 through a high-speed filtering device (namely, a high-speed filtering engine in the invention), and transmitting the filtered data to a message queue system through the data transmission device 1. Corresponding to the data in the data acquisition device 2, the data is transmitted to a high-performance filtering device (namely, a high-performance filtering engine in the invention) through the data transmission device 2, the data in the data acquisition device 2 is filtered through the high-performance filtering device, and the filtered data is transmitted to a message queue system through the data transmission device 3. And finally, processing the data in the message queue system to provide real-time recommendation and real-time feedback for the data of the user behaviors under different scene requirements such as real-time products, real-time reports, real-time monitoring and the like.
In an optional embodiment of the present invention, sequentially filtering the first bitstream data through each filtering logic in the first filtering logic set to obtain a first filtered data set including first data of each category, may include:
sending the first streaming data to a different thread group;
and respectively filtering data included in different thread groups through the filtering logic corresponding to each thread group in the first filtering logic set to obtain a first filtering data set including the filtering data corresponding to each thread group.
Specifically, referring to fig. 6, fig. 6 is a schematic diagram of a framework of a data filtering method according to an embodiment of the present invention, and the specific process is as follows:
all high-priority and low-latency data in the Nginx log source are acquired, then different thread groups are adopted for filtering the data, and finally the filtered data of each thread group (thread group 1, thread group 2 and thread group 3 … … thread group n) is sent to the message queue system. The high-speed filtering engine uses a plurality of thread groups, different filtering logics are respectively used for each thread group, each group of each thread group completes the process of 'acquisition + filtering + sending', and the thread groups are not interfered with each other.
In addition, the filtering logic of the data is uniformly managed by the background management system, and the high-speed filtering engine of the device acquires the data filtering logic of each thread by accessing the interface of the background management system. The filtering logic of the data can be dynamically modified in the background and transmitted to a data separation process on the Nginx machine through heartbeat, so that the real-time updating requirement of the data filtering logic is met. Although the high-speed filtering engine can flexibly provide high-priority and low-delay data for subdividing real-time user behaviors for subsequent teams.
The device also introduces a rear high-performance filtering engine. Referring to fig. 7, fig. 7 is a schematic diagram of a framework of another data filtering method according to an embodiment of the present invention, where data with large traffic in a nginnx log source is obtained through a receiving module, then the data is filtered and calculated through a filtering and calculating module, and finally the filtered data is sent to a message queue system through a sending module.
In fig. 7, the nature of this high performance filtering engine is a distributed real-time computing task, running on a large-scale clustered resource pool. This post-positioned high performance filtering engine, developed using a distributed real-world computing framework, can horizontally extend the use of machine resources since it is located outside of Nginx.
The high-performance filtering engine can filter data of real-time user behaviors with large data volume, can use complex filtering logic, and can also dynamically modify the filtering logic in the high-performance data separation engine in the background.
In an optional embodiment of the present invention, after the data is shunted according to a preset shunting rule to obtain first shunting data and second shunting data, the first shunting data and the second shunting data may be further stored in an Nginx log.
Specifically, Ngnix is a high-performance HTTP (HyperText Transfer Protocol) and a reverse proxy program, can run under Windows and Linux operating systems, has the characteristics of low resource occupation, stable performance and strong expansibility, and is usually used as a program running on an HTTP server. In addition, during the running process of the Ngnix program, a running log can be recorded according to the configuration, and the running log contains HTTP requests, HTTP response information and the like. Because the real-time user behaviors occur on each user end and each user end transmits the real-time user behaviors to the back end based on the HTTP requests, the HTTP requests are received by using the Nginx, and the user behavior data is recorded in the Nginx log, so that the occupied memory is small, and the concurrency capability is strong.
Correspondingly, the filtering the first streaming data by the first data filtering engine corresponding to the first streaming data to obtain a first filtered data set including the first data of each category, and filtering the second streaming data by the second data filtering engine corresponding to the second streaming data to obtain a second filtered data set including the second data of each category, includes:
and filtering second shunt data in the Nginx log through a second data filtering engine corresponding to the second shunt data to obtain a second filtered data set containing second data of each category.
Generally, when a server runs for a period of time, the amount of data stored in the log becomes large and is not easy to find and analyze. The Nginx server has a log cutting function, the current log is renamed and stored, then the server reloads the configuration file, and the log file is regenerated. Therefore, after the first streaming data and the second streaming data are respectively stored in the Nginx log, the first streaming data and the second streaming data in the Nginx log are respectively filtered, and the accuracy of data filtering is improved.
Referring to fig. 8, fig. 8 is a schematic structural diagram of a data processing apparatus for real-time user behavior according to an embodiment of the present invention, including the following modules:
an obtaining module 801, configured to obtain data of a real-time user behavior;
the shunting module 802 is configured to shunt the data according to a preset shunting rule to obtain first shunting data and second shunting data;
a filtering module 803, configured to filter the first bitstream data through a first data filtering engine corresponding to the first bitstream data to obtain a first filtered data set including first data of each category, and filter the second bitstream data through a second data filtering engine corresponding to the second bitstream data to obtain a second filtered data set including second data of each category;
the processing module 804 is configured to respond to all data in the first filtered data set and all data in the second filtered data set, determine each first message to be pushed corresponding to each category of first data and each second message to be pushed corresponding to each category of second data, and push each first message to be pushed and each second message to be pushed to a corresponding user.
Therefore, according to the data processing device for the real-time user behavior provided by the embodiment of the invention, the data of the real-time user behavior is obtained through the obtaining module; then the shunting module shunts the data according to a preset shunting rule to obtain first shunting data and second shunting data; the filtering module filters the first streaming data through a first data filtering engine corresponding to the first streaming data to obtain a first filtering data set containing first data of each category, and filters the second streaming data through a second data filtering engine corresponding to the second streaming data to obtain a second filtering data set containing second data of each category; and finally, responding all data in the first filtered data set and all data in the second filtered data set through the processing module, determining all first messages to be pushed corresponding to all kinds of first data and all second messages to be pushed corresponding to all kinds of second data, and pushing all the first messages to be pushed and all the second messages to be pushed to corresponding users. The data of all real-time user behaviors are shunted firstly, and then the shunted data of the real-time user behaviors are filtered respectively, so that the flexibility of real-time recommendation system change is met, the real-time user behaviors are fed back quickly, and the user experience is improved.
Further, the apparatus further comprises:
the first storage module is used for storing all data in the first filtered data set and all data in the second filtered data set into the message queue system;
correspondingly, the processing module 804 is specifically configured to:
all data in the first filtered data set and all data in the second filtered data set are obtained from the message queue system, all data in the first filtered data set and all data in the second filtered data set are responded, first messages to be pushed corresponding to the first data of each category and second messages to be pushed corresponding to the second data of each category are determined, and the first messages to be pushed and the second messages to be pushed are pushed to corresponding users.
Further, the shunting module 802 includes:
the first obtaining submodule is used for obtaining the priority, delay and flow of each data in the data;
the first streaming submodule is used for streaming data of which the priority is higher than the preset priority and the delay is lower than the preset delay in the data to obtain first streaming data containing first data of each category;
and the second shunting submodule is used for shunting the data with the flow larger than the preset threshold value in the data to obtain second shunting data containing second data of each category.
Further, the filtering module 803 includes:
the second obtaining submodule is used for obtaining a first filtering logic set containing all filtering logics in the first data filtering engine and a second filtering logic set containing all filtering logics in the second data filtering engine;
the first filtering submodule is used for sequentially filtering the first streaming data through each filtering logic in the first filtering logic set to obtain a first filtering data set containing first data of various types;
and the second filtering submodule is used for sequentially filtering the second sub-stream data through each filtering logic in the second filtering logic set to obtain a second filtering data set containing second data of each category.
Further, the first filtering submodule includes:
a sending unit for sending the first streaming data to different thread groups;
and the filtering unit is used for filtering the data included in different thread groups respectively through the filtering logics of the thread groups corresponding to the first filtering logic set to obtain a first filtering data set containing the filtering data corresponding to each thread group.
Further, the apparatus further comprises:
a second storage module to store the first streaming data and the second streaming data in a Nginx log;
accordingly, the filtering module 803 is specifically configured to:
and filtering second shunt data in the Nginx log through a second data filtering engine corresponding to the second shunt data to obtain a second filtered data set containing second data of each category.
The embodiment of the present invention further provides an electronic device, as shown in fig. 9, which includes a processor 901, a communication interface 902, a memory 903 and a communication bus 904, where the processor 901, the communication interface 902, and the memory 903 complete mutual communication through the communication bus 904.
A memory 903 for storing computer programs;
the processor 901 is configured to implement the data processing method of the real-time user behavior described in any of the above embodiments when executing the program stored in the memory 903.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the device can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, a discrete Gate or transistor logic device, or a discrete hardware component.
Therefore, according to the electronic device provided by the embodiment of the invention, the data of the real-time user behavior is acquired through the acquisition module; then the shunting module shunts the data according to a preset shunting rule to obtain first shunting data and second shunting data; the filtering module filters the first streaming data through a first data filtering engine corresponding to the first streaming data to obtain a first filtering data set containing first data of each category, and filters the second streaming data through a second data filtering engine corresponding to the second streaming data to obtain a second filtering data set containing second data of each category; and finally, responding all data in the first filtered data set and all data in the second filtered data set through the processing module, determining all first messages to be pushed corresponding to all kinds of first data and all second messages to be pushed corresponding to all kinds of second data, and pushing all the first messages to be pushed and all the second messages to be pushed to corresponding users. The data of all real-time user behaviors are shunted firstly, and then the shunted data of the real-time user behaviors are filtered respectively, so that the flexibility of real-time recommendation system change is met, the real-time user behaviors are fed back quickly, and the user experience is improved.
In another embodiment of the present invention, a computer-readable storage medium is further provided, which stores instructions that, when executed on a computer, cause the computer to perform a data processing method for real-time user behavior as described in any one of the above embodiments.
Therefore, according to the computer-readable storage medium provided by the embodiment of the invention, the data of the real-time user behavior is acquired through the acquisition module; then the shunting module shunts the data according to a preset shunting rule to obtain first shunting data and second shunting data; the filtering module filters the first streaming data through a first data filtering engine corresponding to the first streaming data to obtain a first filtering data set containing first data of each category, and filters the second streaming data through a second data filtering engine corresponding to the second streaming data to obtain a second filtering data set containing second data of each category; and finally, responding all data in the first filtered data set and all data in the second filtered data set through the processing module, determining all first messages to be pushed corresponding to all kinds of first data and all second messages to be pushed corresponding to all kinds of second data, and pushing all the first messages to be pushed and all the second messages to be pushed to corresponding users. The data of all real-time user behaviors are shunted firstly, and then the shunted data of the real-time user behaviors are filtered respectively, so that the flexibility of real-time recommendation system change is met, the real-time user behaviors are fed back quickly, and the user experience is improved.
In yet another embodiment of the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform a method for real-time user behavioural data processing as described in any one of the above embodiments.
Therefore, according to the computer program product provided by the embodiment of the invention, the data of the real-time user behavior is acquired through the acquisition module; then the shunting module shunts the data according to a preset shunting rule to obtain first shunting data and second shunting data; the filtering module filters the first streaming data through a first data filtering engine corresponding to the first streaming data to obtain a first filtering data set containing first data of each category, and filters the second streaming data through a second data filtering engine corresponding to the second streaming data to obtain a second filtering data set containing second data of each category; and finally, responding all data in the first filtered data set and all data in the second filtered data set through the processing module, determining all first messages to be pushed corresponding to all kinds of first data and all second messages to be pushed corresponding to all kinds of second data, and pushing all the first messages to be pushed and all the second messages to be pushed to corresponding users. The data of all real-time user behaviors are shunted firstly, and then the shunted data of the real-time user behaviors are filtered respectively, so that the flexibility of real-time recommendation system change is met, the real-time user behaviors are fed back quickly, and the user experience is improved.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the embodiments of the apparatus, the electronic device, and the computer-readable storage medium, since they are substantially similar to the embodiments of the method, the description is simple, and for the relevant points, reference may be made to the partial description of the embodiments of the method.
Claims (11)
1. A method for processing data of real-time user behavior, the method comprising:
acquiring data of real-time user behaviors;
shunting the data according to a preset shunting rule to obtain first shunting data and second shunting data;
filtering the first streaming data through a first data filtering engine corresponding to the first streaming data to obtain a first filtered data set containing first data of each category, and filtering the second streaming data through a second data filtering engine corresponding to the second streaming data to obtain a second filtered data set containing second data of each category;
storing all data in the first filtered data set and all data in the second filtered data set in a message queue system;
and acquiring all data in the first filtered data set and all data in the second filtered data set from the message queue system, responding to all data in the first filtered data set and all data in the second filtered data set, determining each first message to be pushed corresponding to each category of first data and each second message to be pushed corresponding to each category of second data, and pushing each first message to be pushed and each second message to be pushed to corresponding users.
2. The method according to claim 1, wherein the splitting the data according to a preset splitting rule to obtain first splitting data and second splitting data comprises:
acquiring the priority, delay and flow of each data in the data;
shunting data of which the priority is higher than a preset priority and the delay is lower than a preset delay in the data to obtain first shunting data;
and shunting the data with the flow larger than a preset threshold value in the data to obtain second shunting data.
3. The method of claim 1, wherein the filtering the first bitstream data by a first data filtering engine corresponding to the first bitstream data to obtain a first filtered data set including first data of each category, and filtering the second bitstream data by a second data filtering engine corresponding to the second bitstream data to obtain a second filtered data set including second data of each category, comprises:
acquiring a first filtering logic set containing all filtering logic in the first data filtering engine and a second filtering logic set containing all filtering logic in the second data filtering engine;
sequentially filtering the first streaming data through each filtering logic in the first filtering logic set to obtain the first filtering data set containing first data of each category;
and sequentially filtering the second sub-stream data through each filtering logic in the second filtering logic set to obtain a second filtering data set containing second data of each category.
4. The method of claim 3, wherein said filtering the first bitstream data in turn by each filtering logic of the first set of filtering logics to obtain the first set of filtered data comprising first data of each category, comprises:
sending the first streaming data to a different thread group;
and respectively filtering data included in different thread groups through the filtering logic corresponding to each thread group in the first filtering logic set to obtain the first filtering data set including the filtering data corresponding to each thread group.
5. The method according to any one of claims 1 to 4, wherein after the splitting the data according to a preset splitting rule to obtain first split data and second split data, the method further comprises:
storing the first and second streaming data in a Nginx log;
correspondingly, the filtering the first streaming data by the first data filtering engine corresponding to the first streaming data to obtain a first filtered data set including first data of each category, and filtering the second streaming data by the second data filtering engine corresponding to the second streaming data to obtain a second filtered data set including second data of each category, includes:
and filtering the first streaming data in the Nginx log through a first data filtering engine corresponding to the first streaming data to obtain a first filtered data set containing first data of each category, and filtering the second streaming data in the Nginx log through a second data filtering engine corresponding to the second streaming data to obtain a second filtered data set containing second data of each category.
6. A data processing apparatus for real-time user behavior, the apparatus comprising:
the acquisition module is used for acquiring data of real-time user behaviors;
the shunting module is used for shunting the data according to a preset shunting rule to obtain first shunting data and second shunting data;
the filtering module is used for filtering the first streaming data through a first data filtering engine corresponding to the first streaming data to obtain a first filtered data set containing first data of each category, and filtering the second streaming data through a second data filtering engine corresponding to the second streaming data to obtain a second filtered data set containing second data of each category;
a first storage module, configured to store all data in the first filtered data set and all data in the second filtered data set in a message queue system;
and the processing module is used for acquiring all data in the first filtered data set and all data in the second filtered data set from the message queue system, responding to all data in the first filtered data set and all data in the second filtered data set, determining each first message to be pushed corresponding to each category of first data and each second message to be pushed corresponding to each category of second data, and pushing each first message to be pushed and each second message to be pushed to corresponding users.
7. The apparatus of claim 6, wherein the flow diversion module comprises:
the first obtaining submodule is used for obtaining the priority, delay and flow of each data in the data;
the first streaming submodule is used for streaming data of which the priority is higher than a preset priority and the delay is lower than a preset delay in the data to obtain first streaming data containing first data of each category;
and the second shunting submodule is used for shunting the data of which the flow is greater than a preset threshold value in the data to obtain second shunting data containing second data of each category.
8. The apparatus of claim 6, wherein the filtration module comprises:
a second obtaining sub-module, configured to obtain a first filtering logic set including all filtering logic in the first data filtering engine and a second filtering logic set including all filtering logic in the second data filtering engine;
the first filtering submodule is used for sequentially filtering the first streaming data through each filtering logic in the first filtering logic set to obtain the first filtering data set containing first data of each category;
and the second filtering submodule is used for sequentially filtering the second sub-stream data through each filtering logic in the second filtering logic set to obtain the second filtering data set containing the second data of each category.
9. The apparatus of claim 8, wherein the first filtering submodule comprises:
a sending unit, configured to send the first streaming data to different thread groups;
and the filtering unit is used for filtering data included in different thread groups respectively through the filtering logics of the thread groups corresponding to the first filtering logic set to obtain the first filtering data set containing the filtering data corresponding to each thread group.
10. The apparatus according to any one of claims 6-9, further comprising:
a second storage module to store the first and second streaming data in a Nginx log;
correspondingly, the filtering module is specifically configured to:
and filtering the first streaming data in the Nginx log through a first data filtering engine corresponding to the first streaming data to obtain a first filtered data set containing first data of each category, and filtering the second streaming data in the Nginx log through a second data filtering engine corresponding to the second streaming data to obtain a second filtered data set containing second data of each category.
11. An electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus;
the memory is used for storing a computer program;
the processor, when executing the program stored in the memory, implementing the method steps of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710650747.5A CN107566450B (en) | 2017-08-02 | 2017-08-02 | Data processing method and device for real-time user behavior and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710650747.5A CN107566450B (en) | 2017-08-02 | 2017-08-02 | Data processing method and device for real-time user behavior and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107566450A CN107566450A (en) | 2018-01-09 |
| CN107566450B true CN107566450B (en) | 2020-08-28 |
Family
ID=60974929
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710650747.5A Active CN107566450B (en) | 2017-08-02 | 2017-08-02 | Data processing method and device for real-time user behavior and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107566450B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103078876A (en) * | 2013-01-31 | 2013-05-01 | 北京集奥聚合科技有限公司 | Method and device for protecting privacy of HTTP (hyper text transport protocol) message |
| CN104023072A (en) * | 2014-06-13 | 2014-09-03 | 中国民航信息网络股份有限公司 | Device and method for pushing data |
| CN105049906A (en) * | 2015-08-07 | 2015-11-11 | 虎扑（上海）文化传播股份有限公司 | Data processing method and electronic device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201322022A (en) * | 2011-11-24 | 2013-06-01 | Alibaba Group Holding Ltd | Distributed data stream processing method |
-
2017
- 2017-08-02 CN CN201710650747.5A patent/CN107566450B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103078876A (en) * | 2013-01-31 | 2013-05-01 | 北京集奥聚合科技有限公司 | Method and device for protecting privacy of HTTP (hyper text transport protocol) message |
| CN104023072A (en) * | 2014-06-13 | 2014-09-03 | 中国民航信息网络股份有限公司 | Device and method for pushing data |
| CN105049906A (en) * | 2015-08-07 | 2015-11-11 | 虎扑（上海）文化传播股份有限公司 | Data processing method and electronic device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107566450A (en) | 2018-01-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10834218B2 (en) | Event information system classifying messages using machine learning classification model and pushing selected message to user | |
| US9727613B2 (en) | Distributed data stream processing method and system | |
| US11816172B2 (en) | Data processing method, server, and computer storage medium | |
| CN104951465B (en) | Application recommendation method and device | |
| US20160277485A1 (en) | Socially driven feed of aggregated content in substantially real time | |
| US20170185454A1 (en) | Method and Electronic Device for Determining Resource Consumption of Task | |
| CN110020062B (en) | Customizable web crawler method and system | |
| US11423096B2 (en) | Method and apparatus for outputting information | |
| WO2017092582A1 (en) | Data processing method and apparatus | |
| CN109063158B (en) | Method, device, system and medium for inquiring website access ranking information | |
| CN104219230B (en) | Identify method and the device of malicious websites | |
| CN111104583B (en) | Live broadcast room recommendation method, storage medium, electronic equipment and system | |
| CN110659404A (en) | Information recommendation method and device and storage medium | |
| WO2013169179A2 (en) | Social intelligence architecture | |
| KR102009020B1 (en) | Method and apparatus for providing website authentication data for search engine | |
| CN114223189A (en) | Duration statistical method and device, electronic equipment and computer readable medium | |
| US20160275193A1 (en) | Socially driven feed of aggregated content in substantially real time | |
| CN114281648A (en) | Data acquisition method and device, electronic equipment and storage medium | |
| US20170337192A1 (en) | Network node, indexing server and methods performed thereby for supportingindexing of audio visual content | |
| CN107870921B (en) | Log data processing method and device | |
| CN105245394A (en) | Method and equipment for analyzing network access log based on layered approach | |
| CN107566450B (en) | Data processing method and device for real-time user behavior and electronic equipment | |
| WO2018205642A1 (en) | Video revenue calculation modeling device and method, video recommendation device and method, server, and storage medium | |
| CN112434215B (en) | Ranking list generation method and device, electronic equipment and storage medium | |
| CN110674386B (en) | Resource recommendation method, device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |