CN111913977B - Data processing method, device and medium - Google Patents
Data processing method, device and medium Download PDFInfo
- Publication number
- CN111913977B CN111913977B CN202010835566.1A CN202010835566A CN111913977B CN 111913977 B CN111913977 B CN 111913977B CN 202010835566 A CN202010835566 A CN 202010835566A CN 111913977 B CN111913977 B CN 111913977B
- Authority
- CN
- China
- Prior art keywords
- user
- data
- time
- ranking
- score
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/70—Game security or game management aspects
- A63F13/79—Game security or game management aspects involving player-related data, e.g. identities, accounts, preferences or play histories
- A63F13/798—Game security or game management aspects involving player-related data, e.g. identities, accounts, preferences or play histories for assessing skills or for ranking players, e.g. for generating a hall of fame
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
- G06F16/2255—Hash tables
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Business, Economics & Management (AREA)
- Computer Security & Cryptography (AREA)
- General Business, Economics & Management (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present application relates to the technical field of game software engineering, and in particular, to a method, an apparatus, and a medium for processing data. The dictionary includes one or more field-value pairs, the user identifier and data update time of the user are stored as fields and values, respectively, in the same field-value pair, the ordered set includes one or more member-score pairs, and the user's sort key and user data are stored as members and scores, respectively, in the same member-score pair. The problem that the game ranking list can not simultaneously meet the requirements of supporting trillion values, accurate time recording, high performance and availability and billions of people on a list in the prior art is solved, and a solution which supports transverse expansion of redis clusters, ultra-large values and accurate time stamps, high performance and availability, no invasion and distributed ranking list is realized.
Description
Technical Field
The present application relates to the technical field of game software engineering, and in particular, to a method, an apparatus, and a medium for processing data.
Background
In large-scale online games, a ranking list is a widely applied system, and due to high-frequency interaction of games, the ranking list must be high-performance and high-availability so as not to cause a player to get stuck or fail in operation, and in addition, ranking is an important honor of the player, and can influence resource acquisition of the player, so that reliability of ranking list data is guaranteed. And the total number of players of the popular game reaches hundreds of millions, so that the situation that hundreds of millions of players are on a same list needs to be supported.
Since the game is a virtual world, the range of game values may be large, and the leaderboard score needs to support a range of trillion levels or even greater. And the specific time for the player to reach the point needs to be recorded, firstly, the player can be shown on a game interface, secondly, the player who reaches the point first can be ranked in front as the ranking basis of the same point of the player.
Redis is an open-source memory database, has the characteristics of high performance and high availability, and a built-in Sorted sets data structure (zset) is very suitable for storing ranking list data, but the data type of the fraction used for ranking by Sorted sets is a double-precision floating point number, and when the data type is used for storing integers, only 53-bit significant digits can be accurately stored in binary. As described above, the basis for player ranking is the points and the point achievement time, so the solution common in the industry is as follows: the above formula represents dividing the 53-bit significant number of the fraction of the signed sets into two, the first m bits representing the integral, the last 53-m bits representing the timestamp, the timestamp may represent the integral achievement time, but this scheme has the disadvantage that if the timestamp needs 32 bits to support precise time, the integral only remains 21 bits, and the range of integral support is only 2-21-1, i.e., more than two hundred thousand. If trillion levels of integration are to be supported, the integration takes 44 bits, the time stamp is only 9 bits, can only be used for representing relative time, can only support more than 500 seconds, and cannot meet the requirement at all.
In addition, there are also existing schemes in which the points and the time stamps are stored in the relational database and are sorted after being spliced, but since the performance of the relational database is inferior to that of redis, the scheme cannot satisfy ten thousand ranking list refreshing per second, and a service party is required to additionally process access of the relational data, which has an influence on service development efficiency.
In addition to the above, none of the current leaderboard solutions basically support lateral expansion, both are non-distributed leaderboard systems.
Supporting trillion values, accurate time records, high performance and high availability, and billions of people on a list are practical demands of game on a list, but the universal scheme in the industry is not used for solving the problem that trillion values and accurate time records cannot be fully achieved, and the demand of high performance and high availability of billions of people on a list is difficult to achieve from zero to the realization of the trillion values and accurate time records on a list.
Disclosure of Invention
The application aims to provide a data processing method, equipment and medium, which solve the problem that a game ranking list can not simultaneously meet the requirements of supporting trillion values, accurate time recording, high performance and availability and billions of people in the prior art, and realize a solution which supports the transverse expansion of a redis cluster, has the final consistency of a distributed system, supports oversized values and accurate time stamps, has high performance, is high in availability, can be transversely expanded, and is non-invasive and distributed.
The embodiment of the application discloses a data processing method, wherein the data comprises a user identifier of a user, user data of the user, a sorting key of the user and data updating time of the user data, and the method comprises the following steps:
when the user data and the data updating time of the user need to be updated, searching the data updating time of the user according to the user identifier of the user;
if the data updating time of the user is not found, calculating the sorting key of the user according to the user identifier of the user and the new data updating time, storing the sorting key of the user and the new user data into a second map, and storing the user identifier of the user and the new data updating time into a first map as a field-value pair;
if the data updating time of the user is found, calculating the sorting key of the user according to the user identifier of the user and the found data updating time, deleting the sorting key and the corresponding user data, calculating the new sorting key of the user according to the user identifier of the user and the new data updating time, storing the new sorting key of the user and the new user data into the second map, and replacing the data updating time of the user in the first map with the new data updating time.
Optionally, the first mapping is a dictionary, the dictionary comprising one or more field-value pairs, the user identifier and the data update time of the user being stored as field and value respectively in the same field-value pair.
Optionally, the method further comprises:
the dictionary is implemented by using a Redis hash, wherein the field corresponds to a field of the Redis hash, and the value corresponds to a value of the Redis hash.
Optionally, the second mapping is an ordered set, where the ordered set includes one or more member-score pairs, where the member-score pairs are arranged according to their own scores in a certain order, and the member-score pairs with the same scores are arranged according to their own members in a certain order, and the sorting key of the user and the user data are stored as members and scores in the same member-score pair, respectively.
Optionally, the method further comprises:
the ordered set is implemented by a Redis ordered set, wherein the members correspond to members of the Redis ordered set, and the score corresponds to a score of the Redis ordered set.
Optionally, in the ordered set, the member-score pairs are arranged in order of their own score values from high to low, and the member-score pairs with the same score are arranged in order of their own dictionary.
Optionally, the data update time is expressed in the form of a Unix timestamp.
Optionally, the method further comprises:
setting a maximum time;
defining a time score for a user as the result of subtracting the data update time for the user from the maximum time;
for a user, the time score of the user is on the left, the user identifier of the user is on the right, the time score of the user and the user identifier of the user are connected, and the result of the connection serves as the ranking key of the user.
Optionally, the method further comprises:
and when the time score is defined, carrying out high-order zero padding on the time score so that the number of bits of the time score is consistent with the number of bits of the maximum time.
Optionally, the method further comprises:
when the new user data and the new data updating time of the user need to be updated into the dictionary and the ordered set, the new data updating time of the user is stored in a Redis cluster;
deleting the pre-update data update time of the user after storing the new sort key and the new user data of the user in the second map;
if a network has a problem that one user has two different data updating times, combining the user identifiers, searching the user data according to the two different data updating times, deleting the data updating time which cannot be used for searching the user data, and reserving the data updating time which can be used for searching the user data.
Optionally, the method further comprises:
when the ranking, the user data and the data updating time of the user are required to be obtained according to the user identifier of the user, searching the data updating time of the user according to the user identifier of the user;
if the data updating time of the user is not found, returning information of 'the user is not in the list';
if the data updating time of the user is found, calculating the sorting key of the user according to the user identifier and the data updating time of the user, searching the user data of the user according to the sorting key of the user, determining the ranking of the user, and finally returning the searched user data, the data updating time and the determined ranking.
Optionally, the method further comprises:
when the user identifier, the data updating time and the user data corresponding to the ranking are required to be obtained according to the ranking, searching the ordering key corresponding to the ranking according to the ranking;
if the ranking key corresponding to the ranking is not found, returning information of 'users without the ranking';
if the ranking key corresponding to the ranking is found, calculating the user identifier and the data updating time according to the found ranking key, and returning the user data corresponding to the found ranking key and the calculated user identifier and the calculated data updating time.
An embodiment of the application discloses a data processing apparatus, characterized in that the apparatus comprises a memory storing computer executable instructions and a processor configured to execute the instructions to implement a data processing method.
An embodiment of the present application discloses a computer storage medium encoded with a computer program, wherein the computer program includes instructions that are executed by one or more computers to implement a method of processing data.
Compared with the prior art, the embodiment of the application has the main differences and effects that:
in the application, a user ordering key is defined according to a user identifier and a data updating time of a user, an ordered set comprises one or more member-score pairs, wherein the member-score pairs are arranged according to a certain sequence according to own scores, the member-score pairs with the same scores are arranged according to a certain sequence according to own members, the ordering key and user data of the user are respectively used as members and scores to be stored in the same member-score pair, the data updating time is expressed in a Unix timestamp mode, a maximum time is set, the time score of the user is defined as a result of subtracting the data updating time of the user from the maximum time, the time score of the user is on the left, the user identifier of the user is on the right, the time score of the user and the user identifier of the user are connected, and the connected result is used as the ordering key of the user. The method can be applied to Redis, the ordered set is realized by the Redis ordered set, wherein members correspond to the members of the Redis ordered set, the score corresponds to the score of the Redis ordered set, the 53-bit effective number of the score of the Redis ordered set can be used for storing user data, the value storage of millions of levels is achieved, the data updating time is stored in a sorting key, the sorting key is stored in the members of the Redis ordered set, and the maximum time can be set randomly according to requirements, so that accurate time recording in any time range is supported.
In the application, the dictionary is realized by using the Redis hash, wherein the field corresponds to the field of the Redis hash, the value corresponds to the value of the Redis hash, the ordered set is realized by using the Redis ordered set, wherein the members correspond to the members of the Redis ordered set, the fraction corresponds to the fraction of the Redis ordered set, the Redis database is only used for storing data, a service party does not need to store additional data or perform additional operation during development, high encapsulation is realized, no invasion is caused to service development, and more than 40 hundred million members can be stored by virtue of the high-performance and high-availability characteristics of the Redis, and the high-performance (one hundred thousand updates per second) high availability of a ranking list is realized (data main and standby storage, data cannot be lost by downtime) and the same ranking list of people is supported.
When new user data and new data update time of a user need to be updated into a dictionary and an ordered set, in a Redis cluster, the new data update time of the user is stored first, after a new ordering key and new user data of the user are stored in a second mapping, the data update time before the update of the user is deleted, if a network has a problem so that one user has two different data update times, the user data is searched according to the two different data update times respectively by combining a user identifier, the data update time which cannot be used for searching the user data is deleted, and the data update time which can be used for searching the user data is reserved. And supporting transverse expansion and realizing the final consistency of the distributed system.
Drawings
Fig. 1 shows a flow chart of a method of processing data according to an embodiment of the application.
FIG. 2 shows a schematic diagram of a dictionary and ordered collection according to an embodiment of the present application.
FIG. 3 shows a flow chart for obtaining a ranking of a user, user data, data update time from a user identifier of the user, according to an embodiment of the application.
FIG. 4 shows a flow chart of obtaining a user identifier, a data update time, and user data corresponding to a rank according to the rank according to an embodiment of the application.
Detailed Description
The application will be further described with reference to specific examples and figures. It is to be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application. Furthermore, for convenience of description, only some, but not all, structures or processes related to the present application are shown in the drawings. It should be noted that in the present specification, like reference numerals and letters denote like items in the following drawings.
It will be understood that, although the terms "first," "second," etc. may be used herein to describe various features, these features should not be limited by these terms. These terms are used merely for distinguishing and are not to be construed as indicating or implying relative importance. For example, a first feature may be referred to as a second feature, and similarly a second feature may be referred to as a first feature, without departing from the scope of the example embodiments.
In the description of the present application, it should also be noted that, unless explicitly specified and limited otherwise, the terms "disposed," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present embodiment can be understood in a specific case by those of ordinary skill in the art.
Illustrative embodiments of the application include, but are not limited to, data processing methods, apparatus, and media.
Various aspects of the illustrative embodiments will be described using terms commonly employed by those skilled in the art to convey the substance of their work to others skilled in the art. However, it will be apparent to those skilled in the art that some alternative embodiments may be practiced using the features described in part. For purposes of explanation, specific numbers and configurations are set forth in order to provide a more thorough understanding of the illustrative embodiments. However, it will be apparent to one skilled in the art that the alternative embodiments may be practiced without the specific details. In some other instances, well-known features are omitted or simplified in order not to obscure the illustrative embodiments of the application.
Furthermore, various operations will be described as multiple discrete operations, in a manner that is most helpful in understanding the illustrative embodiments; however, the order of description should not be construed as to imply that these operations are necessarily order dependent, and that many of the operations be performed in parallel, concurrently or with other operations. Furthermore, the order of the operations may also be rearranged. When the described operations are completed, the process may be terminated, but may also have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, and the like.
References in the specification to "one embodiment," "an illustrative embodiment," etc., indicate that the embodiment described may include a particular feature, structure, or property, but every embodiment may or may not necessarily include the particular feature, structure, or property. Moreover, such phrases are not necessarily referring to the same embodiment. Furthermore, when a particular feature is described in connection with a particular embodiment, it is within the knowledge of one skilled in the art to affect such feature in connection with other embodiments, whether or not such embodiment is explicitly described.
The terms "comprising," "having," and "including" are synonymous, unless the context dictates otherwise. The phrase "a and/or B" means "(a), (B) or (a and B)".
As used herein, the term "module" may refer to, be part of, or include: memory (shared, dedicated, or group) for running one or more software or firmware programs, an Application Specific Integrated Circuit (ASIC), an electronic circuit and/or processor (shared, dedicated, or group), a combinational logic circuit, and/or other suitable components that provide the described functionality.
In the drawings, some structural or methodological features may be shown in a particular arrangement and/or order. However, it should be understood that such a particular arrangement and/or ordering is not required. Rather, in some embodiments, these features may be described in a different manner and/or order than shown in the illustrative figures. Additionally, the inclusion of a structural or methodological feature in a particular drawing does not imply that all embodiments need to include such feature, and in some embodiments may not be included or may be combined with other features.
For the purpose of making the objects, technical solutions and advantages of the present application more apparent, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
Fig. 1 shows a flow chart of a method of processing data according to an embodiment of the application.
As shown in fig. 1, the method 100 includes:
step 102, searching the data updating time of the user according to the user identifier of the user;
step 104, judging whether the data updating time of the user can be found;
if the data update time of the user is not found, calculating a sorting key of the user according to the user identifier of the user and the new data update time through step 106, storing the sorting key of the user and the new user data into a second map through step 108, and storing the user identifier of the user and the new data update time into a first map as a field-value pair through step 110;
if the data update time of the user is found, the sorting key of the user is calculated according to the user identifier of the user and the found data update time, the sorting key and the corresponding user data are deleted, the new sorting key of the user is calculated according to the user identifier of the user and the new data update time, the new sorting key and the new user data are stored in the second map, and the data update time of the user in the first map is replaced with the new data update time, via step 114, via step 116.
FIG. 2 shows a schematic diagram of a dictionary and ordered collection according to an embodiment of the present application.
As shown in FIG. 2, dictionary 202 includes one or more field-value pairs, where field-value pair 204 represents any one field-value pair, ordered set 206 includes one or more member-score pairs, where member-score pair 208 represents any one member-score pair, where one or more member-score pairs are arranged in an order according to their own scores, and the same-score member-score pairs are arranged in an order according to their own members.
According to some embodiments of the application, the dictionary is implemented with a Redis hash, wherein a field corresponds to a field of the Redis hash and a value corresponds to a value of the Redis hash; the ordered set is implemented by a Redis ordered set, wherein the members correspond to members of the Redis ordered set and the scores correspond to scores of the Redis ordered set.
The application only uses the Redis database to store data, and the service side does not need to store additional data or carry out additional operation during development, thus realizing high encapsulation, no invasion to service development, and being capable of storing more than 40 hundred million members by virtue of the characteristics of high performance and high availability of the Redis itself and the single Redis ordered set, realizing high performance (updating hundred thousand times per second) high availability of a ranking list (data is stored in a main and standby mode, and data is not lost by downtime), and supporting the same ranking list of hundred million people.
According to some embodiments of the present application, the data includes a user identifier of the user, user data of the user, a ranking key of the user, and a data update time of the user data, the ranking key of the user is defined according to the user identifier of the user and the data update time, the dictionary includes one or more field-value pairs, the user identifier of the user and the data update time are respectively stored in the same field-value pair as a field and a value, the ordered set includes one or more member-score pairs, wherein the member-score pairs are arranged according to their own scores in a certain order, the member-score pairs with the same scores are arranged according to their own scores in a certain order, and the ranking key of the user and the user data are respectively stored in the same member-score pair as a member and a score.
In connection with fig. 2, in the context of dictionary 202 and any one of field-value pairs 204 included therein, ordered collection 206 and any one of member-score pairs 208 included therein, when, for example, a leaderboard is required to be generated for a plurality of user data (e.g., player points), a user identifier (e.g., ID) of each user (e.g., player) is stored in field-value pair 204 as a field of field-value pair 204, a user data update time (e.g., point achievement time) is stored in field-value pair 204 as a value of field-value pair 204, for each user, a ranking key of the user is defined based on the user identifier and the data update time of the user, and the ranking key of each user is stored in member-score pair 208 as a member of member-score pair 208 and the user data is stored as a score of member-score pair 208. The ordered set is used as the realization of the user data ranking list.
Referring to fig. 2, in the environment of the dictionary 202 and the ordered set 206, for the data processing method according to the embodiment of the present application, for example, after the data of the user is updated, new user data and new data update time of the user need to be updated into the dictionary 202 and the ordered set 206, and the data update time of the user is searched in the dictionary 202 according to the user identifier of the user; if the data updating time of the user cannot be found in the dictionary 202, calculating a sorting key of the user according to the user identifier of the user and the new data updating time, storing the sorting key of the user and the new user data as member-score pairs into the ordered set 206, and storing the user identifier of the user and the new data updating time as field-value pairs into the dictionary 202; if the data update time of the user is found in the dictionary 202, the sorting key of the user is calculated according to the user identifier of the user and the found data update time, the sorting key and the corresponding user data are deleted, then the new sorting key of the user is calculated according to the user identifier of the user and the new data update time, then the new sorting key of the user and the new user data are stored in the ordered set 206 as member-score pairs, and then the data update time of the user in the dictionary 202 is replaced by the new data update time.
According to some embodiments of the application, the method further comprises:
in the ordered set, the member-score pairs are arranged in the order of the values of the scores of the member-score pairs from high to low, and the member-score pairs with the same scores are arranged in the dictionary order of the members of the member-score pairs.
Referring to fig. 2, taking dictionary 202 and any one of field-value pairs 204 included therein, ordered set 206 and any one of member-score pairs 208 included therein as an environment, for example, if the user identifier of user a is 100, user data 3000, data update time is 1598327733, the user identifier of user B is 101, user data is 3001, data update time is 194967295, the user identifier of user C is 1001, user data is 3000, data update time is 4967295, the user identifier and data update time of user a are stored in the form of any field-value pair 204 in dictionary 202, the field is user identifier 100, the value is data update time 1598327733, the user identifier and data update time of user B are stored in the form of any field-value pair 204 in dictionary 202, the fields are user identifier 101, the values are data update time 194967295, user identifier of user C and data update time are stored in dictionary 1001 in the form of any field-value pair 204, the fields are user identifier 100, the values are data update time 4967295, user A's sort key and user data are stored in ordered set 206 in the form of any member-score pair 208, the members are sort key 2696639562100, the scores are user data 3000, user B's sort key and user data are stored in ordered set 206 in the form of any member-score pair 208, the members are sort key 4100000000101, the scores are user data 3001, user C's sort key and user data are stored in ordered set 206 in the form of any member-score pair 208, the members are sort key 42900000001001, and the scores are user data 3000. Wherein the order of the member-score pairs of the three users in the ordered set 206 is in turn: a member-score pair for user B, a member-score pair for user C, a member-score pair for user a.
The application can be applied to Redis, the ordered set is realized by the Redis ordered set, wherein the members correspond to the members of the Redis ordered set, the scores correspond to the scores of the Redis ordered set, the 53-bit effective digits of the scores of the Redis ordered set can be completely used for storing user data, the numerical value storage of the level of megahundred million is achieved, the data updating time is stored in a sorting key, the sorting key is stored in the members of the Redis ordered set, and the maximum time can be set randomly according to the requirement, so that the accurate time record of any time range is supported.
According to some embodiments of the application, the data update time is represented in the form of a Unix timestamp.
For example, if the data update time is expressed in Beijing time as: 2020-08-25-11:55:33, its representation in the method is 1598327733.
According to some embodiments of the application, the method further comprises:
setting a maximum time;
defining a time score of a user as a result of subtracting a data update time of the user from a maximum time;
for a user, the time score of the user is on the left, the user identifier of the user is on the right, the time score of the user and the user identifier of the user are connected, and the connected result is used as a sorting key of the user.
For example, set the maximum time to 4294967295 (Unix timestamp representation), if the user's user identifier is 100 and the data update time is 1598327733 (Unix timestamp representation), then the user's sort key is 2696639562100.
According to some embodiments of the application, the method further comprises:
when defining the time score, high order zero padding is performed on the time score so that the number of bits of the time score is consistent with the number of bits of the maximum time.
For example, if the user's sort key is 444, then its representation in the method is 0000000444.
In the application, after updating the ranking list (i.e. the ordered set), new data updating time is needed to be saved so as to generate a new ranking key when inquiring next time, and as the data updating time and the ranking key are in two different Redis key-value pairs, and the distribution of the two Redis key-value pairs to the same node cannot be ensured under the condition of Redis cluster, if the network is interrupted at this time, the new ranking key in the ordered set and the old data updating time in the dictionary can appear, so that the ranking key generated when inquiring the ranking list is old, and records cannot be inquired.
To address the above issues, according to some embodiments of the present application, the method further includes:
when new user data and new data updating time of a user need to be updated into a dictionary and an ordered set, the new data updating time of the user is stored in a Redis cluster;
after storing the new sort key and new user data for the user in a second map, deleting the pre-update data update time for the user;
if the network has a problem that one user has two different data updating times, the user identifier is combined to search the user data according to the two different data updating times, the data updating time which cannot be used for searching the user data is deleted, and the data updating time which can be used for searching the user data is reserved.
The application is realized through the Redis cluster, supports the transverse expansion, can finally reach the final consistency of the distributed ranking list system, and can ensure the accuracy of data even if the network fluctuates.
FIG. 3 shows a flow chart for obtaining a ranking of a user, user data, data update time from a user identifier of the user, according to an embodiment of the application.
As shown in fig. 3, the process 300 includes:
step 302, searching the data updating time of the user according to the user identifier of the user;
step 304, judging whether the data updating time of the user can be found;
if the data update time of the user is not found, returning a message of 'the user is not in the list' through step 312;
if the data update time of the user is found, the sorting key of the user is calculated according to the user identifier and the data update time of the user through step 306, then the user data of the user is found according to the sorting key of the user through step 308, the ranking of the user is determined, and finally the found user data, data update time and the determined ranking are returned through step 310.
Referring to fig. 2, taking the dictionary 202 and the ordered set 206 as the environment, for example, when a user or developer requests to obtain the rank, user data and data update time of the user according to the user identifier of the user, responding to the request, searching the data update time of the user in the dictionary 202 according to the user identifier of the user, and if the data update time of the user is not found in the dictionary, returning information of 'the user is not in the list'; if the data updating time of the user is found in the dictionary, a sorting key of the user is calculated according to the user identifier and the data updating time of the user, then the user data of the user is found in the ordered set 206 according to the sorting key of the user, the ranking of the user is determined, and finally the found user data, the data updating time and the determined ranking are returned.
FIG. 4 shows a flow chart of obtaining a user identifier, a data update time, and user data corresponding to a rank according to the rank according to an embodiment of the application.
As shown in fig. 4, the process 400 includes:
step 402, searching the ordering key corresponding to the ranking according to the ranking;
step 404, judging whether the ranking key corresponding to the ranking can be found;
if the ranking key corresponding to the ranking is not found, returning information of "users without the ranking" through step 410;
if the ranking key corresponding to the ranking is found, the user identifier and the data update time are calculated according to the found ranking key in step 406, and then the user data corresponding to the found ranking key and the calculated user identifier and the data update time are returned in step 408.
Referring to fig. 2, in the ordered set 206, for example, when a user or developer requests to obtain a user identifier, data update time, and user data corresponding to the rank according to the rank, in response to the request, a sort key corresponding to the rank is searched in the ordered set 206 according to the rank; if no sort key corresponding to the rank exists in the ordered set 206, returning information of "users without the rank"; if the ranking key corresponding to the ranking exists in the ordered set 206, the user identifier and the user data are calculated according to the searched ranking key, and then the user data corresponding to the searched ranking key and the calculated user identifier and user data are returned.
According to some embodiments of the present application, a data processing apparatus is disclosed, the apparatus comprising a memory storing computer executable instructions and a processor configured to execute the instructions to implement a method of processing data.
The first embodiment is a method embodiment corresponding to the present embodiment, and the present embodiment can be implemented in cooperation with the first embodiment. The related technical details mentioned in the first embodiment are still valid in this embodiment, and in order to reduce repetition, a detailed description is omitted here. Accordingly, the related art details mentioned in the present embodiment can also be applied to the first embodiment.
According to some embodiments of the present application, a computer storage medium encoded with a computer program is disclosed, the computer program comprising instructions that are executed by one or more computers to implement a method of processing data.
The first embodiment is a method embodiment corresponding to the present embodiment, and the present embodiment can be implemented in cooperation with the first embodiment. The related technical details mentioned in the first embodiment are still valid in this embodiment, and in order to reduce repetition, a detailed description is omitted here. Accordingly, the related art details mentioned in the present embodiment can also be applied to the first embodiment.
In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented in the form of instructions or a program loaded onto or stored on one or more transitory or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors, etc. When the instructions or programs are executed by a machine, the machine may perform the various methods described above. For example, the instructions may be distributed over a network or other computer readable medium. Thus, a machine-readable medium may include, but is not limited to, any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), such as a floppy disk, an optical disk, a compact disk read-only memory (CD-ROMs), a magneto-optical disk, a read-only memory (ROM), a Random Access Memory (RAM), an erasable programmable read-only memory (EPROM), an electronically erasable programmable read-only memory (EEPROM), a magnetic or optical card, or a flash memory or a tangible machine-readable memory for transmitting network information via electrical, optical, acoustical or other form of signal (e.g., carrier waves, infrared signals, digital signals, etc.). Thus, a machine-readable medium includes any form of machine-readable medium suitable for storing or transmitting electronic instructions or information readable by a machine (e.g., a computer).
The embodiments of the present application have been described in detail above with reference to the accompanying drawings, but the use of the technical solution of the present application is not limited to the applications mentioned in the embodiments of the present application, and various structures and modifications can be easily implemented with reference to the technical solution of the present application to achieve the various advantageous effects mentioned herein. Various changes, which may be made by those skilled in the art without departing from the spirit of the application, are deemed to be within the scope of the application as defined by the appended claims.
Claims (13)
1. A method for processing data, wherein the data includes a user identifier of a user, user data of the user, a sort key of the user, and a data update time of the user data, the method comprising:
setting a maximum time;
defining a time score for a user as the result of subtracting the data update time for the user from the maximum time;
connecting the time score of the user and the user identifier of the user as the ranking key of the user;
when the user data and the data updating time of the user need to be updated, searching the data updating time of the user according to the user identifier of the user;
if the data updating time of the user is not found, calculating the sorting key of the user according to the user identifier of the user and the new data updating time, storing the sorting key of the user and the new user data into a second map, and storing the user identifier of the user and the new data updating time into a first map as a field-value pair;
if the data updating time of the user is found, calculating the sorting key of the user according to the user identifier of the user and the found data updating time, deleting the sorting key and the corresponding user data, calculating the new sorting key of the user according to the user identifier of the user and the new data updating time, storing the new sorting key of the user and the new user data into the second map, and replacing the data updating time of the user in the first map with the new data updating time.
2. The method of claim 1, wherein the first mapping is a dictionary comprising one or more field-value pairs, the user identifier and data update time of a user being stored as field and value respectively in the same field-value pair.
3. The method according to claim 2, wherein the method further comprises:
the dictionary is implemented by using a Redis hash, wherein the field corresponds to a field of the Redis hash, and the value corresponds to a value of the Redis hash.
4. The method of claim 1, wherein the second map is an ordered set comprising one or more member-score pairs, wherein the member-score pairs are arranged in an order according to their own scores, wherein the member-score pairs having the same score are arranged in an order according to their own members, wherein the ranking key of the user and the user data are stored as member and score pairs, respectively.
5. The method of claim 4, wherein the method further comprises:
the ordered set is implemented by a Redis ordered set, wherein the members correspond to members of the Redis ordered set, and the score corresponds to a score of the Redis ordered set.
6. The method of claim 4, wherein the member-score pairs are arranged in the ordered set in order of higher-to-lower values of scores, and wherein the member-score pairs having the same score are arranged in the dictionary order of members.
7. The method of claim 1, wherein the data update time is represented in the form of a Unix timestamp.
8. The method of claim 7, wherein the method further comprises:
and when the time score is defined, carrying out high-order zero padding on the time score so that the number of bits of the time score is consistent with the number of bits of the maximum time.
9. The method according to claim 1, wherein the method further comprises:
when the new user data and the new data updating time of the user need to be updated into the dictionary and the ordered set, the new data updating time of the user is stored in the Redis cluster;
deleting the pre-update data update time of the user after storing the new sort key and the new user data of the user in the second map;
if a network has a problem that one user has two different data updating times, combining the user identifiers, searching the user data according to the two different data updating times, deleting the data updating time which cannot be used for searching the user data, and reserving the data updating time which can be used for searching the user data.
10. The method according to claim 1, wherein the method further comprises:
when the ranking, the user data and the data updating time of the user are required to be obtained according to the user identifier of the user, searching the data updating time of the user according to the user identifier of the user;
if the data updating time of the user is not found, returning information of 'the user is not in the list';
if the data updating time of the user is found, calculating the sorting key of the user according to the user identifier and the data updating time of the user, searching the user data of the user according to the sorting key of the user, determining the ranking of the user, and finally returning the searched user data, the data updating time and the determined ranking.
11. The method according to claim 1, wherein the method further comprises:
when the user identifier, the data updating time and the user data corresponding to the ranking are required to be obtained according to the ranking, searching the ordering key corresponding to the ranking according to the ranking;
if the ranking key corresponding to the ranking is not found, returning information of 'users without the ranking'; if the ranking key corresponding to the ranking is found, calculating the user identifier and the data updating time according to the found ranking key, and returning the user data corresponding to the found ranking key and the calculated user identifier and the calculated data updating time.
12. A data processing apparatus, characterized in that the apparatus comprises a memory storing computer executable instructions and a processor configured to execute the instructions to implement a method of processing data according to any of claims 1-11.
13. A computer storage medium encoded with a computer program, the computer program comprising instructions that are executed by one or more computers to implement the method of processing data according to any one of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010835566.1A CN111913977B (en) | 2020-08-19 | 2020-08-19 | Data processing method, device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010835566.1A CN111913977B (en) | 2020-08-19 | 2020-08-19 | Data processing method, device and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111913977A CN111913977A (en) | 2020-11-10 |
| CN111913977B true CN111913977B (en) | 2023-10-20 |
Family
ID=73278837
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010835566.1A Active CN111913977B (en) | 2020-08-19 | 2020-08-19 | Data processing method, device and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111913977B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112698926B (en) * | 2021-03-25 | 2021-07-02 | 成都新希望金融信息有限公司 | Data processing method, device, equipment, storage medium and system |
| CN113194339A (en) * | 2021-05-20 | 2021-07-30 | 广州虎牙科技有限公司 | Live list generation method and device, electronic equipment and readable storage medium |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102740143A (en) * | 2012-07-03 | 2012-10-17 | 合一网络技术(北京)有限公司 | Network video ranking list generation system based on user behavior and method thereof |
| CN108898428A (en) * | 2018-06-19 | 2018-11-27 | 努比亚技术有限公司 | A kind of terminal user enlivens determination method, server and the storage medium of index |
| CN110300311A (en) * | 2019-07-01 | 2019-10-01 | 腾讯科技（深圳）有限公司 | Battle method, apparatus, equipment and storage medium in live broadcast system |
| CN110413633A (en) * | 2018-04-28 | 2019-11-05 | 武汉斗鱼网络科技有限公司 | Inactive users screening technique and readable medium and equipment based on Redis database |
| CN110427394A (en) * | 2019-08-08 | 2019-11-08 | 北京字节跳动网络技术有限公司 | Data manipulation method and device |
| CN110674180A (en) * | 2019-09-26 | 2020-01-10 | 腾讯科技（深圳）有限公司 | Business data processing method and device and readable storage medium |
| CN110975296A (en) * | 2019-11-25 | 2020-04-10 | 网易（杭州）网络有限公司 | Method, device, equipment and storage medium for maintaining ranking list in game |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10292040B2 (en) * | 2013-03-29 | 2019-05-14 | Roamware, Inc. | Methods and apparatus for facilitating LTE roaming between home and visited operators |
-
2020
- 2020-08-19 CN CN202010835566.1A patent/CN111913977B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102740143A (en) * | 2012-07-03 | 2012-10-17 | 合一网络技术(北京)有限公司 | Network video ranking list generation system based on user behavior and method thereof |
| CN110413633A (en) * | 2018-04-28 | 2019-11-05 | 武汉斗鱼网络科技有限公司 | Inactive users screening technique and readable medium and equipment based on Redis database |
| CN108898428A (en) * | 2018-06-19 | 2018-11-27 | 努比亚技术有限公司 | A kind of terminal user enlivens determination method, server and the storage medium of index |
| CN110300311A (en) * | 2019-07-01 | 2019-10-01 | 腾讯科技（深圳）有限公司 | Battle method, apparatus, equipment and storage medium in live broadcast system |
| CN110427394A (en) * | 2019-08-08 | 2019-11-08 | 北京字节跳动网络技术有限公司 | Data manipulation method and device |
| CN110674180A (en) * | 2019-09-26 | 2020-01-10 | 腾讯科技（深圳）有限公司 | Business data processing method and device and readable storage medium |
| CN110975296A (en) * | 2019-11-25 | 2020-04-10 | 网易（杭州）网络有限公司 | Method, device, equipment and storage medium for maintaining ranking list in game |
Non-Patent Citations (2)
| Title |
|---|
| NoSql system for big data management;Dhana Rao等;《2014 IEEE world cngress on services》;1-9 * |
| 用户兴趣模型驱动的个性化搜索引擎的研究和实现;王苏;《中国优秀硕士学位论文全文数据库 信息科技辑》;I138-1627 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111913977A (en) | 2020-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110147407B (en) | Data processing method and device and database management server | |
| US10585915B2 (en) | Database sharding | |
| CN111913977B (en) | Data processing method, device and medium | |
| CN106709042B (en) | Index updating method and equipment | |
| CN111475519B (en) | Data caching method and device | |
| CN110830234A (en) | User traffic distribution method and device | |
| CN110597852A (en) | Data processing method, device, terminal and storage medium | |
| CN113495872A (en) | Transaction processing method and system in distributed database | |
| CN111544895A (en) | Ranking list processing method and device, electronic equipment and storage medium | |
| CN112685237A (en) | Chip test data tracking query method, system and electronic equipment | |
| CN110652728A (en) | Game resource management method and device, electronic equipment and storage medium | |
| CN111400301A (en) | Data query method, device and equipment | |
| CN110765125B (en) | Method and device for storing data | |
| CN111290714A (en) | Data reading method and device | |
| CN109753506B (en) | Data distributed storage method, device, terminal and storage medium | |
| CN115914387A (en) | Resource sharing and data query method, device and server | |
| CN112988913B (en) | Data processing method and related device | |
| CN113688148A (en) | Urban rail data query method and device, electronic equipment and readable storage medium | |
| CN117009430A (en) | Data management method, device, storage medium and electronic equipment | |
| CN111399759B (en) | Method for reading data and writing data and object file system | |
| CN113905252A (en) | Data storage method and device for live broadcast room, electronic equipment and storage medium | |
| CN108984615B (en) | Data query method and system and storage medium | |
| CN113282617A (en) | Data query method and business system page turning method | |
| CN115185984A (en) | Hbase database query node optimization method, device, equipment and storage medium | |
| KR100286750B1 (en) | Device and Method for Error Detection in Exchange Database Management System |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Gu Jianye Inventor after: Xu Jianguo Inventor after: Shao Zhenyu Inventor after: Yang Sheng Inventor before: Gu Jianye Inventor before: Xu Jianguo Inventor before: Shao Zhenyu Inventor before: Yang Cheng |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |