WO2017219740A1

WO2017219740A1 - Method and system for maintaining active user set based on time roulette and page heartbeat

Info

Publication number: WO2017219740A1
Application number: PCT/CN2017/080776
Authority: WO
Inventors: 吴瑞诚
Original assignee: 武汉斗鱼网络科技有限公司
Priority date: 2016-06-24
Filing date: 2017-04-17
Publication date: 2017-12-28
Also published as: CN106162247B; CN106162247A

Abstract

Disclosed are a method and system for maintaining an active user set based on a time roulette and a page heartbeat, which relate to the technical field of the maintenance of an active user set during live video broadcast. The method involves the steps of: generating video heartbeat information according to a live broadcast video viewed by a user; buffering and pre-processing video heartbeat information meeting a specification to obtain pre-processed page heartbeat data; and updating user IDs, corresponding to all the pre-processed page heartbeat data passing verification, to an active user set fragment corresponding to the current time, and adding the updated active user set fragment to a pre-created time roulette. The present invention can update an active user set by utilizing a time roulette so as to accomplish the timely maintenance of an active user set. Not only is the operation efficiency higher, but a larger load is also not created for the system; moreover, the participation of inactive users can be effectively restricted in certain interactive activities, thereby ensuring that the interactive activities are carried out efficiently.

Description

Active user set maintenance method and system based on time roulette and page heartbeat

Technical field

The present invention relates to the field of maintenance technology for active user sets in live video broadcasts, and in particular, to a method and system for maintaining active user sets based on time roulette and page heartbeat.

Background technique

With the rapid development of Internet technology, more and more users can watch online live video broadcasts through the Internet using terminals such as computers and mobile phones. Online live video broadcast refers to live video live broadcast service using Internet network resources. It is simultaneously posted to the network through live video capture. Users can see the real-time situation on the network at the same time.

In the business scene of the live video website, many live broadcast anchors need to only interact with active users who are watching live broadcasts or active users on the website when launching interactive activities or when the website launches a special event. An active set of users to record and update live users in real time.

Currently, in the live video field, the general idea of maintaining an active user set is: when the server does not receive the page heartbeat event data for a period of time (usually it needs to customize the time, the timeout is Timeout), The user culls the active user set. Specifically, it is usually divided into the following two methods:

(1) For each user, save the corresponding lastReceiveTime (the time when the page heartbeat event data was last received); then, through a timer, traverse all user sessions every second, and eliminate the user session that satisfies the following formula:

now-lastReceiveTime>Timeout, where now is the current time.

The disadvantage of the method (1) is that the method (1) has only one repeated timer globally. (Repetition timer), when the number of users is large (for example, tens of thousands of users are maintained at the same time), the repeated timer traverses all user sessions each time, and the workload is long and the work efficiency is low.

(2) is similar to method (1), the only difference is that method (2) is to set a one-shot timer for each user session, and each one-shot timer receives the corresponding one. When the time of the page heartbeat event data is automatically updated, if each one-shot timer finds a timeout, the corresponding user session is culled.

Although the method (2) improves the efficiency of the check to some extent, there are still disadvantages: the method (2) needs to set a larger number of one-shot timers, and the one-shot timer has a faster frequency. When the number of users is large, the number of connections in the user session is large, which may cause pressure on the "waiting to update the timer queue". In severe cases, the system may be congested or even collapsed.

Summary of the invention

In view of the deficiencies in the prior art, the technical problem solved by the present invention is to provide an active user set maintenance method and system based on time roulette and page heartbeat. The invention can use the time roulette to update the active user set and complete the timely maintenance of the active user set; not only the work efficiency is high, but also does not cause a large load on the system, and can effectively limit the non-active activities in some interactive activities. The participation of active users ensures the effective implementation of interactive activities.

To achieve the above objective, the method for maintaining an active user set based on time roulette and page heartbeat provided by the present invention comprises the following steps:

A, generating video heartbeat information according to the live video watched by the user, the video heartbeat information includes a plurality of heartbeat information identification identifiers, and proceeds to step B;

B, determining the video heartbeat information according to the specified heartbeat information according to the heartbeat information identification, buffering and pre-processing the video heartbeat information that meets the requirements, and obtaining the page heartbeat pre-processing data, and proceeding to step C;

C, the page heartbeat pre-processing data is composed into a plurality of data pieces, each piece of data piece includes at least one page heartbeat pre-processing data; periodically all the currently cached pieces of data are allocated into groups according to a hashing policy, and the process proceeds to step D;

D. Determine the page heartbeat pre-processing data of all the verifications in each group of data slices; update the user ID corresponding to the page heartbeat pre-processing data of all the verifications to the active user set fragment corresponding to the current time, and go to the step E; the active user set fragmentation refers to: a plurality of fragments that are divided into active user sets in advance according to a specified active calculation time period;

E. Timing adds the updated active user set fragment to the pre-created time wheel.

The present invention provides a time-based roulette and page heartbeat-based active user set maintenance system for implementing the above method, the system comprising a video heartbeat information generating module located on each terminal device, a cache pre-processing module located on the server, and located on the server. Data slice component module, several real-time computing modules located on the server, and active user set function modules located on the server;

The video heartbeat information generating module is configured to: generate video heartbeat information according to the live video viewed by the user, where the video heartbeat information includes a plurality of heartbeat information identification identifiers, and the video heartbeat information is submitted to the buffer preprocessing module;

The buffer pre-processing module is configured to: determine, according to the heartbeat information identification identifier, the video heartbeat information that meets the requirements; and after buffering and pre-processing the video heartbeat information that meets the requirements, obtain the page heartbeat pre-processing data, and periodically send the data piece to the data slice component module. Composition signal

The data slice forming module is configured to: after receiving the data slice forming signal, the page heartbeat preprocessing data is formed into a plurality of data pieces, each piece of data piece includes at least one page heartbeat preprocessing data; and all current cached data pieces are periodically timed according to A hashing policy is assigned to each real-time computing module;

The real-time computing module is configured to: verify all page heartbeat pre-processing data in the data piece allocated by the data piece forming module, determine all the page heartbeat pre-processing data passed by the verification; and all the users corresponding to the page heartbeat pre-processing data passed through the verification The ID is updated to the active user set fragment corresponding to the current time; the active user set fragment refers to: a plurality of fragments into which the active user set is divided into the specified active computing time period in advance;

The active user set function module is used to: periodically add the updated user set fragment of the real-time calculation module to the pre-created time wheel.

The advantages of the present invention over the prior art are:

(1) The present invention generates video heartbeat information according to the live video viewed by the user, and updates the video heartbeat information that is verified as an active user to the corresponding active user set fragment. In view of this, compared with the prior art, the present invention does not use a repeating timer or a one-time timer, but periodically adds an active user set fragment to the created time wheel to utilize the time roulette. To update the active user set and complete the timely maintenance of the active user set; not only work efficiency is high, but also does not cause a heavy load on the system, and can effectively limit the participation of inactive users in certain interactive activities, and ensure The interactive activities are carried out effectively.

(2) The division time of the active user set fragment of the present invention and the timing period of adding the active user set fragment to the time wheel can be set and adjusted according to the specific use situation; thereby making the active user set fragmented. Quantity, statistical granularity of active users (that is, statistics by hourly activity, minute activity, or other activity), and maintenance schedule of time roulette (maintenance cycle is the same as the timing cycle, can be maintained for 1 hour, 1 Minute maintenance or other cycles to maintain) can be adjusted accordingly. Therefore, the present invention is more flexible and has higher applicability.

(3) The system of the present invention includes a plurality of real-time computing modules for processing data pieces, and the plurality of real-time computing modules can simultaneously process a plurality of data pieces, thereby further improving work efficiency. Timeliness.

DRAWINGS

1 is a flowchart of a method for maintaining an active user set based on a time roulette and a page heartbeat according to an embodiment of the present invention;

2 is a schematic structural diagram of a time wheel in an embodiment of the present invention;

FIG. 3 is a structural block diagram of an active user set maintenance system based on time roulette and page heartbeat.

detailed description

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

Referring to FIG. 1 , a method for maintaining an active user set based on time roulette and page heartbeat in the embodiment of the present invention includes the following steps:

S1: The terminal device used by each user generates video heartbeat information according to the live video viewed by the user, and the video heartbeat information includes a plurality of heartbeat information identification identifiers, and goes to S2.

The heartbeat information identification identifier includes a user ID (ie, an ID of the user uniqueness) and an identification code. The identification code is a fixed length, and is encrypted after being generated. The generation rule of the identification code is: a timestamp, a terminal device ID used by the user, and a random number, wherein the terminal device ID used by the user is based on the API of the terminal device (Application Programming Interface, application programming interface).

S2: Determine, according to the heartbeat information identification identifier, the video heartbeat information that meets the requirements (discard the video heartbeat information that does not meet the requirements), cache and pre-process the video heartbeat information that meets the requirements (that is, the unified data format), and obtain the page heartbeat preprocessing. Data, go to S3.

The heartbeat information identification identifier in the video heartbeat information that meets the requirements in S2 needs to meet the following conditions: the user ID is not empty (it is not compliant if it is empty), and the user ID conforms to the data field type (the data field type does not meet the requirements) ), the timestamp format is positive Indeed (the error is not in compliance with the regulations), the user terminal type identification is legal (illegal does not meet the requirements).

The page heartbeat preprocessing data format in S2 is:

S3: The page heartbeat pre-processing data is formed into a plurality of data pieces, each piece of data piece has a capacity of less than or equal to 1 MB, and each piece of data piece includes at least one complete page heartbeat pre-processing data, and is transferred to S4.

The specific process of S3 is as follows: The current cached page heartbeat preprocessing data is 3, and their sizes are 0.3M, 0.4M, and 0.5M respectively. The process of S3 is: preprocessing two presets of 0.3M and 0.4M. The data constitutes a piece of data, and then the 0.5M preprocessed data is formed into another piece of data, and so on.

S4: Timing all current cached data pieces according to a hash policy to be at least 3 groups, and going to S5. The specific process of S4 is: defining a total number of groups of N, assigning a unique ID number UUID to each piece of data, performing a modulo operation (UUID mod N) for each UUID and N, and obtaining a remainder of the modulo operation All the same pieces of data are in the same group.

The purpose of grouping the data slices according to the hash strategy in S4 is to increase or decrease N according to the number of data slices in the grouping, thereby improving the horizontal expansion capability of each group of data slices.

S5: Verifying all the page heartbeat pre-processing data in each group of data pieces respectively, if the verification is passed, go to S6; if the verification fails, discard the page heartbeat of the failed verification page Preprocess the data and end.

The specific process of S5 is: decrypting the identification codes of all the page heartbeat preprocessing data in each group of data pieces to obtain a timestamp and a terminal device ID; determining whether the timestamp is within a reasonable range (ie, the timestamp and the current server) Whether the time difference is within one minute), and the terminal device ID conforms to the specification (the terminal device ID satisfies the generation rule of the identification code as conforming to the specification), and if so, determines that the current page heartbeat pre-processing data is verified, otherwise the current page heartbeat pre-processing is determined. Data verification failed.

S6: Update the user ID corresponding to the page heartbeat preprocessing data of all the verifications to the active user set fragment corresponding to the current time (that is, the time when the verification passes), and go to S7. Active user set fragmentation refers to a number of fragments in which an active user set is divided into a specified active computing time period in advance.

An example of the active user set fragmentation in S6 is as follows: if the specified active computing time period is 1 hour, the active user set corresponds to 24 hours a day, and there are 24 corresponding active user set fragments; if the specified active computing time period is specified For 1 minute, the active user set corresponds to 24*60=1440 minutes, and there are 1440 corresponding active user set fragments.

S7: Timing (the timing period can be set by itself, usually 1 minute) Add the updated active user set fragment to the pre-created time wheel and end.

Referring to FIG. 2, the time wheel in S7 includes a circular data structure buffer loop queue (ie, circular buffer) connected end to end, and the ring data structure buffer loop queue is divided into several unit slots, and each unit slot is filled with one piece. The active user set is fragmented; the circular data structure buffering loop queue is provided with a pointer to the tail unit slot.

The S7 specifically includes the following process: defining a clockwise direction in the time roulette from the end of the team to the head of the team, and a counterclockwise direction from the head to the end of the team. After the timeout period, the active user set in the first unit slot of the time roulette is removed, and the active user sets in each of the remaining unit slots are moved clockwise to the next unit slot (at the end of the queue) of The cell slot is empty). The updated active user set fragment in S6 is added to the unit slot at the end of the time roulette, and the pointer is moved clockwise to the next unit slot.

The time roulette (active user set) in S7 is exposed to other applications through the Restful interface or RPC interface. Based on this active user set, the users participating in the interactive activity can be guaranteed to be currently active users in a plurality of important scenarios.

As shown in FIG. 3, the time-based roulette and page heartbeat-based active user set maintenance system for implementing the foregoing method in the embodiment of the present invention includes a video heartbeat information generating module located on each terminal device, and a cache pre-location on the server. A processing module, a data slice component module located on the server, a number of real-time computing modules located on the server, and an active user set function module located on the server.

The video heartbeat information generating module is configured to: generate video heartbeat information according to the live video viewed by the user, where the video heartbeat information includes a plurality of heartbeat information identification identifiers, and the video heartbeat information is submitted to the buffer preprocessing module. The heartbeat information identification identifier includes a user ID and an identification code, and the identification code is a fixed length, and is encrypted after being generated; the generation rule of the identification code is: a timestamp, a terminal device ID used by the user, and a random number.

The buffer pre-processing module is configured to: determine, according to the heartbeat information identification identifier, the video heartbeat information that meets the requirements, cache and pre-process the video heartbeat information that meets the requirements, obtain the page heartbeat pre-processing data, and periodically send the data slice to the data slice component module. Composition of the signal. The heartbeat information identification identifier in the video heartbeat information that meets the requirements must meet the following conditions: the user ID is not empty, the user ID matches the data field type, the timestamp format is correct, and the user terminal type identifier is legal.

The data slice forming module is configured to: after receiving the data slice forming signal, the page heartbeat preprocessing data is formed into a plurality of data pieces, each piece of data piece includes at least one page heartbeat preprocessing data; and all current cached data pieces are periodically timed according to The hash policy is assigned to each real-time computing module. The specific process is: define the total number of groups as N, and allocate each piece of data. A unique ID number UUID; each UUID and N are modulo-operated, and all data pieces with the same remainder obtained by the modulo operation are the same group.

The real-time computing module is configured to: verify all the page heartbeat pre-processing data in the data piece allocated by the data piece forming module, and the specific process is: decrypting the identification code of the page heartbeat pre-processing data in all the allocated data pieces, and obtaining Timestamp and terminal device ID. Determining whether the timestamp is within a reasonable range, and the terminal device ID conforms to the specification. If not, determining that the current page heartbeat pre-processing data verification fails, discarding the current page heartbeat pre-processing data; if yes, determining that the current page heartbeat pre-processing data is verified, The user ID corresponding to the page heartbeat pre-processing data of all the verifications is updated to the active user set fragment corresponding to the current time; the active user set fragmentation means that the active user set is divided according to the specified active calculation time period in advance. A number of fragments.

The active user set function module is used to: periodically add the updated user set fragment of the real-time calculation module to the pre-created time wheel. The time roulette includes a circular data structure buffering circular queue connected end to end. The circular data structure buffering circular queue is divided into a plurality of unit slots, each unit slot is filled with an active user set fragmentation; the ring data structure buffering loop queue is set. There is a pointer to the slot unit slot.

On this basis, the active user set function module is specifically used to: after the timeout period, the active user set in the first unit slot of the time roulette squad is removed, and the active user sets in each of the other unit slots are respectively segmented according to Move clockwise to the next unit slot, clockwise from the end of the team to the head of the team, counterclockwise from the head to the end of the team; add the updated active user set slice of the real-time calculation module to the time roulette team The tail unit slot moves the pointer clockwise to the next unit slot.

The present invention is not limited to the above embodiments, and those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. These improvements and retouchings are also considered as protection of the present invention. Within the scope. Not detailed in this manual The description is of the prior art known to those skilled in the art.

Claims

An active user set maintenance method based on time roulette and page heartbeat, characterized in that the method comprises the following steps:

A, generating video heartbeat information according to the live video watched by the user, the video heartbeat information includes a plurality of heartbeat information identification identifiers, and proceeds to step B;

B, determining the video heartbeat information according to the specified heartbeat information according to the heartbeat information identification, buffering and pre-processing the video heartbeat information that meets the requirements, and obtaining the page heartbeat pre-processing data, and proceeding to step C;

C, the page heartbeat pre-processing data is composed into a plurality of data pieces, each piece of data piece includes at least one page heartbeat pre-processing data; periodically all the currently cached pieces of data are allocated into groups according to a hashing policy, and the process proceeds to step D;

D. Determine the page heartbeat pre-processing data of all the verifications in each group of data slices; update the user ID corresponding to the page heartbeat pre-processing data of all the verifications to the active user set fragment corresponding to the current time, and go to the step E; the active user set fragmentation refers to: a plurality of fragments that are divided into active user sets in advance according to a specified active calculation time period;

E. Timing adds the updated active user set fragment to the pre-created time wheel.
The method for maintaining an active user set based on time roulette and page heartbeat according to claim 1, wherein the heartbeat information identification identifier in step A comprises a user ID and an identification code, and the identification code is a fixed length. Encryption is performed; the generation rule of the identification code is: the time stamp, the terminal device ID used by the user, and the random number are arranged;

On the basis of this, the heartbeat information identification identifier in the video heartbeat information that meets the requirements in step B needs to satisfy the following conditions: the user ID is not empty, the user ID conforms to the data field type, the timestamp format is correct, and the user terminal type identifier legitimate.
Time-based roulette and page heartbeat based active user set as claimed in claim The maintenance method is characterized in that: determining the page heartbeat pre-processing data of all the verifications in each group of data pieces in the step D, specifically comprising the following process: respectively identifying the identification code of the heartbeat pre-processing data of all the pages in each group of data pieces Decrypting, obtaining a timestamp and a terminal device ID; determining whether the timestamp is within a reasonable range, and the terminal device ID conforms to the specification; if yes, determining that the current page heartbeat preprocessing data is verified, otherwise determining that the current page heartbeat preprocessing data verification fails , discard the current page heartbeat preprocessing data.
The time roulette and page heartbeat based active user set maintenance method according to any one of claims 1 to 3, wherein in step C, all currently cached data pieces are allocated according to a hash policy. The specific process of the group is as follows: the total number of groups is defined as N, and each piece of data is assigned a unique ID number UUID; each UUID and N are subjected to modulo operation, and all pieces of data with the same remainder are obtained by modulo operation For the same group.
The time-based roulette and page heartbeat-based active user set maintenance method according to any one of claims 1 to 3, wherein the time wheel in step E comprises a ring-shaped data structure buffering circular queue connected end to end. The ring data structure buffering loop queue is divided into a plurality of unit slots, each unit slot is filled with an active user set fragment; and the ring data structure buffering loop queue is provided with a pointer to the tail unit slot;

On this basis, step E specifically includes the following process: defining a clockwise direction in the time roulette from the end of the team to the head of the team, and a counterclockwise direction from the head to the end of the team; after the timeout period, the first of the time roulette squadrons The active user set fragments in the unit slot are removed, and the active user set fragments in each of the remaining unit slots are moved clockwise to the next unit slot; the updated active user set fragments in step D are added to The unit slot at the end of the time roulette moves the pointer clockwise to the next unit slot.
A time roulette and page heartbeat based active user set maintenance system implementing the method of any one of claims 1 to 5, characterized in that the system comprises a video heartbeat information generating module on the terminal device, a cache preprocessing module located on the server, a data slice forming module located on the server, a plurality of real-time computing modules located on the server, and an active user set function module located on the server;

The video heartbeat information generating module is configured to: generate video heartbeat information according to the live video viewed by the user, where the video heartbeat information includes a plurality of heartbeat information identification identifiers, and the video heartbeat information is submitted to the buffer preprocessing module;

The buffer pre-processing module is configured to: determine, according to the heartbeat information identification identifier, the video heartbeat information that meets the requirements, cache and pre-process the video heartbeat information that meets the requirements, obtain the page heartbeat pre-processing data, and periodically send the data slice to the data slice component module. Composition signal

The data slice forming module is configured to: after receiving the data slice forming signal, the page heartbeat preprocessing data is formed into a plurality of data pieces, each piece of data piece includes at least one page heartbeat preprocessing data; and all current cached data pieces are periodically timed according to A hashing policy is assigned to each real-time computing module;

The real-time computing module is configured to: verify all page heartbeat pre-processing data in the data piece allocated by the data piece forming module, determine all the page heartbeat pre-processing data passed by the verification; and all the users corresponding to the page heartbeat pre-processing data passed through the verification The ID is updated to the active user set fragment corresponding to the current time; the active user set fragment refers to: a plurality of fragments into which the active user set is divided into the specified active computing time period in advance;

The active user set function module is used to: periodically add the updated user set fragment of the real-time calculation module to the pre-created time wheel.
The time-based roulette and page heartbeat-based active user set maintenance system according to claim 6, wherein the heartbeat information identification identifier in the video heartbeat information obtained by the video heartbeat information generating module includes a user ID and an identification code. The identification code is a fixed length, and is encrypted after being generated; the generation rule of the identification code is: a timestamp, a terminal device ID used by the user, and a random number;

On the basis of the above, the heartbeat information identification identifier in the compliant video heartbeat information determined by the buffer pre-processing module needs to satisfy the following conditions: the user ID is not empty, the user ID conforms to the data field type, the timestamp format is correct, and the user The terminal type identifier is legal.
The time-based roulette and page heartbeat-based active user set maintenance system according to claim 7, wherein the real-time computing module verifies all page heartbeat pre-processing data in the data slice allocated by the data piece composition module. The specific process is: decrypting the identification code of the page heartbeat preprocessing data in all the allocated data pieces, obtaining the timestamp and the terminal device ID, determining whether the timestamp is within a reasonable range, and the terminal device ID conforms to the specification, and if so, It is determined that the current page heartbeat pre-processing data is verified, otherwise it is determined that the current page heartbeat pre-processing data verification fails, and the current page heartbeat pre-processing data is discarded.
The time-based roulette and page heartbeat-based active user set maintenance system according to any one of claims 6 to 8, wherein the data slice component module allocates all currently cached data pieces into groups according to a hash policy. The specific process is as follows: the total number of groups is defined as N, and each piece of data is assigned a unique ID number UUID; each UUID and N are subjected to modulo operation, and all data pieces with the same remainder obtained by modulo operation are the same. group.
The time-based roulette and page heartbeat-based active user set maintenance system according to any one of claims 6 to 8, wherein the time roulette in the active user set function module comprises a ring-shaped data structure buffer loop connected end to end. Queue, ring data structure buffer loop queue is divided into several unit slots, each unit slot is filled with an active user set fragment; a circular data structure buffer loop queue is set with a pointer to the tail unit slot;

On this basis, the active user set function module is specifically used to: after the timeout period, the active user set in the first unit slot of the time roulette squad is removed, and the active user sets in each of the other unit slots are respectively segmented according to Move clockwise to the next unit slot, in time The direction of the needle is from the end of the team to the head of the team, and the counterclockwise direction is from the head to the end of the team; the active user set fragment updated by the real-time calculation module is added to the unit slot at the end of the time roulette, and the pointer is clockwise. Move to the next unit slot.