CN112000885A - List service system, list generation method, server and storage medium - Google Patents
List service system, list generation method, server and storage medium Download PDFInfo
- Publication number
- CN112000885A CN112000885A CN202010820861.XA CN202010820861A CN112000885A CN 112000885 A CN112000885 A CN 112000885A CN 202010820861 A CN202010820861 A CN 202010820861A CN 112000885 A CN112000885 A CN 112000885A
- Authority
- CN
- China
- Prior art keywords
- list
- word frequency
- search
- local
- storage service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 186
- 230000008569 process Effects 0.000 claims abstract description 141
- 238000004891 communication Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims 1
- 230000002159 abnormal effect Effects 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000005457 optimization Methods 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9538—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/216—Parsing using statistical methods
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Probability & Statistics with Applications (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The invention discloses a list service system, a list generation method, a server and a storage medium, wherein the system comprises: the system comprises at least one list service process and at least one storage service point, wherein a local word frequency data table and a local list related to the associated list service process are stored in the storage service point; other list lists related to other storage service points are also stored; and each list service process generates a current list according to the local word frequency data table, the local list and each other list in the corresponding storage service point and sends the current list to the corresponding storage service point. By using the system, each storage service point and the list service process can participate in the generation of the list, the flexible deployment of the whole distributed architecture for list generation is realized, the high availability of the system is improved, the complex operation of switching the main node of the existing system under the abnormal condition is avoided, and the data consistency and the correctness of the system are better embodied.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a list service system, a list generation method, a server, and a storage medium.
Background
The hot search list is a key for reflecting content search heat change in the live broadcast platform, and mainly shows search objects with higher search volume and higher heat of a user and corresponding hot search degrees. Generally, a list service system may increase a search word frequency corresponding to a search word through a search of the search word by a user, so as to determine whether the search word reaches a ranking of a list according to the search word frequency. The information in the list can be dynamically displayed to the user, so that other users are attracted to search the word, and the exposure rate of the information related to the hot searched word on the list is increased. In addition, the operator can also achieve the promotion and drainage effect on the specified object through the artificial configuration of the hot search list displayed in the product.
In order to keep the accuracy of word frequency calculation, a main node is usually selected and is specially responsible for explaining the search results of a user, then word frequency increment is added to the locally stored hot searched word frequency, and the hot searched word formed by calculation is synchronized to all other slave nodes, so that the same hot searched word finally has the same word frequency on all the nodes.
However, the system architecture of the conventional list service system is write-once read-many, and write traffic in the whole system is all concentrated on the master node, so that system expansion is not easy to realize, meanwhile, since the slave nodes need to completely depend on the master node for data updating, the list data is inaccurate, once the master node fails, a set of complex operations need to be relied on for master node selection and switching to ensure the reliability and data correctness of the system, and the whole realization cost is relatively high.
Disclosure of Invention
The embodiment of the invention provides a list service system, a list generation method, a server and a storage medium, and effectively improves the data consistency, high availability and expandability of the list service system.
In a first aspect, an embodiment of the present invention provides a list service system, including: the system comprises at least one list service process and at least one storage service point, wherein communication connection exists between the storage service points, and the storage service points are in communication connection with the associated list service process;
the storage service point stores a local word frequency data table and a local list related to the associated list service process; other list lists relevant to other storage service points are stored;
and each list service process is used for generating a current list according to the local word frequency data table, the local list and other list lists acquired from the corresponding storage service point, sending the current list to the corresponding storage service point to replace the original local list and storing the current list as a new local list.
In a second aspect, an embodiment of the present invention provides a list generating method, which is executed by the list service system provided in the first aspect of the present embodiment, and includes:
the list service process acquires a local word frequency data list, a local list and other list lists from the corresponding storage service points;
and the list service process generates a current list according to the acquired local word frequency data list, the local list and other list lists, and sends the current list to a corresponding storage service point to serve as a new local list.
In a third aspect, an embodiment of the present invention provides a server, where the server is used as a storage service point in a list service system provided in the first aspect of the present invention and/or as an execution terminal of a list service process in the list service system provided in the first aspect of the present invention, and the server includes:
one or more processors;
storage means for storing one or more programs;
the one or more programs are executed by the one or more processors, so that the one or more processors are used for executing the list generating method of the second aspect of the invention
In the above-described list service system, list generation method, server, and storage medium, the list service system includes: the system comprises at least one list service process and at least one storage service point, wherein communication connection exists between the storage service points, and the storage service points are in communication connection with the associated list service process; the storage service point stores a local word frequency data table and a local list related to the associated list service process; other list lists relevant to other storage service points are stored; and each list service process is used for generating a current list according to the local word frequency data table, the local list and other list lists acquired from the corresponding storage service point, sending the current list to the corresponding storage service point to replace the original local list and storing the current list as a new local list. In the list service system and the related technical scheme provided by the embodiment, a distributed system architecture without a main node is adopted, so that each storage service point and a list service process can participate in the generation of the list, the writability and readability of all nodes under the distributed architecture are ensured, the flexible deployment of the whole distributed architecture for list generation is realized, and the high availability of the list service system is improved; meanwhile, the complex operation that the existing system needs to switch the main node under the abnormal condition is avoided. In addition, the list generation of the list service system provided by the implementation avoids the influence of single room error data on the whole situation to the maximum extent, and better embodies the data consistency and the accuracy of the list service system.
Drawings
Fig. 1 is an architecture diagram of a list service system according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a list generating method according to a second embodiment of the present invention;
fig. 3 is a flowchart illustrating an example of implementing list generation based on the list service system in the second embodiment of the present invention;
fig. 4 is a schematic diagram of a hardware structure of a server according to a third embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are only some embodiments of the invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. In addition, the embodiments and features of the embodiments of the present invention may be combined with each other without conflict, and the respective embodiments may be mutually referred to and cited.
Example one
Fig. 1 is an architecture diagram of a list service system according to an embodiment of the present invention, and as shown in fig. 1, the list service system includes: the system comprises at least one list service process 11 and at least one storage service point 12, wherein communication connection exists between the storage service points 12, and the storage service points 12 are in communication connection with the associated list service process 11;
the storage service point 12 stores a local word frequency data table and a local list related to the associated list service process 11; other list lists related to other storage service points 12 are also stored;
each list service process 11 is configured to generate a current list according to the local word frequency data table, the local list and each other list obtained from the corresponding storage service point 12, and send the current list to the corresponding storage service point 12 to replace the original local list, so as to store the current list as a new local list.
In this embodiment, the list service system may be specifically understood as a list service support system deployed at a background server of the live broadcast application, and the list service system may be specifically considered as a distributed architecture system, and may be specifically implemented on background servers installed in different regions to ensure global service deployment. The different regions actually correspond to a server cluster called a machine room, and each server cluster contains a background server with a certain scale.
In this embodiment, both the storage service point 12 and the list service process 11 may be regarded as an execution mechanism for implementing list generation in the system, the storage service point 12 may be specifically assumed by one server in the distributed architecture system, and the number of the storage service points 12 included in the entire list service system may be the same as the number of machine rooms included in the distributed architecture system, that is, one storage service point 12 may be correspondingly present in one machine room, and each storage service point 12 stores at least relevant data for list generation. Meanwhile, the list service process 11 may be specifically regarded as a service process created on any background server in each computer room for list generation. Each list service process 11 provided in the same room is associated with a storage service point 12 provided in the room, and establishes communication connection.
Specifically, the list service system according to the present embodiment corresponds to a system capable of dynamically generating a list by mutual communication between the storage service points 12 and mutual communication between the list service process 11 and the associated storage service points 12. In the system, a local word frequency data table, a local list and other list lists which need to be dynamically updated are specifically stored on the storage service point 12, wherein the local word frequency data table and the local list are updated according to data information (word frequency increment data or a newly produced list) fed back by the list service process 11 associated with the storage service point 12, the other list lists are actually equivalent to the local list lists stored on the other storage service points 12, and the storage service point 12 can receive the local list lists fed back by the other storage service points 12 to store the local list lists as the other list lists. In addition, in the present embodiment, the local word frequency data table and the local list in each storage service point 12 may initially default to empty.
Meanwhile, the list service process 11 updates and generates the list through the data information stored in the associated storage service point 12, and specifically, the list service process 11 may acquire the local word frequency data table, the local list and other list lists stored in the associated storage service point 12 to generate the latest current list. The list service processes 11 associated with the same storage service point 12 share the local word frequency data table, the local list and other list lists stored in the storage service point 12, and the operations of the list service processes 11 associated with the same storage service point 12 are independent from each other, so that the updating and storing of related data on the associated storage service points 12 are not influenced.
It should be noted that the generation of the list in the present system needs to be realized by the storage service point 12 and the associated list service process 11 in cooperation with other storage service points 12, for example, when the list service process 11 generates the list, the storage service point 12 needs to update the local list by receiving the current list generated by the associated list service process 11 and update the updated local list to other storage service points 12 in order to update other list lists on other storage service points 12 according to the local list, the local word frequency data table, and other list lists stored in the associated storage service point 12. For another example, the local word frequency data table required by the list service process 11 for generating the list is updated depending on the word frequency increment data provided by the list service process 11.
In the list service system provided by the first embodiment of the invention, a distributed system architecture without a main node is adopted, so that each storage service point 12 and the list service process 11 can participate in the generation of the list, the writing and readability of all nodes under the distributed architecture are ensured, the flexible deployment of the whole distributed architecture for list generation is realized, and the high availability of the list service system is improved; meanwhile, the complex operation that the existing system needs to switch the main node under the abnormal condition is avoided. In addition, the list generation of the list service system provided by the implementation avoids the influence of single room error data on the whole situation to the maximum extent, and better embodies the data consistency and the accuracy of the list service system.
Further, the system may further include: and the search service process 13 is configured to respond to a search request sent by the user terminal, and send a search result to any associated target list service process 11 when the search result of a search term associated with the search request is determined to be non-empty.
In this embodiment, the system further includes a search service process 13 required for list generation, where the search service process 13 is also created on a server in each machine room in the distributed architecture system, and may serve as a search response interface of a user, receive a search request sent by a user terminal side, and feed back a search result for the user, and meanwhile, may also establish a communication connection with each list service process 11 in the same machine room, so as to implement data interaction with each list service process 11.
Specifically, the search service process 13 may receive a search request sent by a user through a user terminal, search a search word included in the search request, and perform search result feedback; when the search result corresponding to the search term is determined to be non-empty, one associated list service process 11 may be selected and the search term corresponding to the search request is sent to the list service process 11, where the list service process 11 selected by the search service process 13 for sending the search term is recorded as the target list service process 11 in the present embodiment.
It can be understood that one application scenario of the list service system provided in this embodiment is a live broadcast application, and in the live broadcast application, the search performed by the user through the user terminal may be a search for a name of a main broadcast, a search for a room number of the main broadcast, or a search for a live broadcast topic when the main broadcast is live broadcast. The keyword sent to the target list service process 11 in this embodiment is mainly a keyword in the search request, and this embodiment is recorded as a search term.
It should be noted that, in the embodiment, the search service process 13 is adopted to select any one of the list service processes 11 to perform feedback of the search term, so that a situation that all write traffic of the whole network is concentrated on the master node is effectively avoided, and therefore, the list service system provided by the embodiment has a lateral expansion capability.
Further, in the present system, the target list service process 11 is further configured to determine, after receiving the search term, word frequency increment data of the search term, and feed back the search term and the corresponding word frequency increment data to the associated storage service point 12.
In this embodiment, the target list service process 11 that receives the search term sent by the search service process 13 may count the number of searches that receive the search term at the current time (there is a case where a plurality of search service processes 13 send the same search term to one list service process 11 at the same time), and use the counted number of searches of the search term as the word frequency increment data of the search term; then, the target list service process 11 needs to feed back the search term and the word frequency increment data to the associated storage service point 12, so as to update the local word frequency data table stored thereon through the storage service point 12.
On the basis of the optimization, the storage service points 12 corresponding to the target list service process 11 are further configured to update the stored local word frequency data table according to the search terms and the corresponding word frequency increment data, and synchronize the search terms and the corresponding word frequency increment data to other storage service points 12;
and the other storage service points 12 are configured to update the corresponding stored local word frequency data table according to the received search word and the corresponding word frequency increment data.
In this embodiment, the storage service point 12 corresponding to the target list service process 11 may receive the feedback of the target list service process 11 on the search term and the corresponding word frequency increment data, and then it may directly update the local word frequency data table stored thereon according to the received search term and the word frequency increment data; and after the storage service point 12 implements local update, in order to ensure global consistency of data, the storage service point 12 further synchronizes the search word and the corresponding word frequency increment to other storage service points 12. Other storage service points 12 may also update their locally stored local word frequency data tables according to the received search word and corresponding word frequency increment data.
It can be understood that the data information transmitted by the storage service point 12 associated with the target list service process 11 to other storage service points 12 in the system is the search word and the word frequency increment data corresponding to the search word, and the operation of synchronizing the word frequency increment data instead of the word frequency data per se performed in this embodiment can effectively avoid the situation that data is abnormal due to storage abnormality in one storage service point 12 in the system, so that all data in the whole network is polluted.
For example, assuming that the word frequency data recorded for the search word a in the storage service point a is 200, and the word frequency data recorded for the search word a by the storage service point B due to the storage service failure is 100, if only the feedback of the word frequency increment data (e.g., +1) corresponding to the search word a is performed from the storage service point B to the other storage service points 12 (including the storage service point a) at this time, the word frequency data corresponding to the search word a after the word frequency update is performed by the storage service point a and the storage service point B based on the word frequency increment data is 201 and 101, respectively, thereby ensuring that the data information stored by the storage service point a is not affected by the storage service point B.
On the basis of the above embodiment, each storage service point 12 may be further specifically configured to:
after receiving the search word and corresponding word frequency increment data, searching whether the search word exists from a stored local word frequency data table; if the search word exists, determining current word frequency data corresponding to the search word in the local word frequency data table, and storing the sum of the current word frequency data and the word frequency increment data in a correlation mode as new current word frequency data; and if the search word does not exist, adding the search word to the local word frequency data table, and storing the word frequency increment data serving as the current word frequency data of the search word in the local word frequency data table in a correlation manner.
This embodiment specifically provides a process for implementing local word frequency data table update by the storage service point 12 based on the search word and the corresponding word frequency increment data. For each storage service point 12, after receiving a search word and corresponding word frequency increment data, it may first search for whether the search word exists from a stored local word frequency data table, and if so, may directly add the received word frequency increment data to the current word frequency data currently corresponding to the search word to form new current word frequency data; if the search word does not exist, the search word can be added into a local word frequency data table, and the currently obtained word frequency increment data is used as initial current word frequency data.
Further, each list service process 11 is further configured to, when receiving a list display request sent by the user terminal, feed back the local list acquired from the corresponding storage service point 12 to the user terminal, so as to display the list through the user terminal.
In this embodiment, the list generated by the system needs to be displayed on the user terminal, the list display request sent by the user terminal is equivalent to one trigger of the list display, and the list service process 11 may directly obtain the local list currently stored in the corresponding storage service point 12 after receiving the list display request, and then send the local list to the user terminal, so that the user terminal may directly display the local list.
Further, the storage service point 12 is configured to receive the current list sent by the corresponding list service process 11, replace the stored local list as a new local list, and synchronize the new local list to each of the other storage service points 12, so that each of the other storage service points serves as a new other list to replace the stored other list.
In this embodiment, there is a communication connection between the storage service points 12, and in order to ensure global consistency of data, when data on the list at one storage service point 12 is changed, real-time update of data stored at other storage service points 12 needs to be performed through the established communication connection, and the stored data also includes other list lists.
On the basis of the optimization, each list service process 11 specifically includes:
an information obtaining sub-process, configured to obtain a local word-frequency data table and a local list currently stored in the corresponding storage service point 12, and other list lists correspondingly fed back by each storage service point 12;
a word frequency statistics sub-process, configured to update current word frequency data of each search word in the local word frequency data table according to the local list and each other list, so as to obtain a new local word frequency data table;
the list generation sub-process is used for generating a current list according to the updated local word frequency data table;
and the information sending sub-process is used for sending the generated current list to the corresponding storage service point 12, so that the storage service point 12 replaces the stored local list with the current list.
The embodiment of the present invention provides a specific implementation that the list service process 11 generates the current list through the local list, the local word frequency data table, and the other list lists. Specifically, the sub-processes with different functions in the list service process 11 cooperate with each other, and processes of acquiring related data information, counting related word frequency information, generating a list, sending list information, and the like are respectively realized.
Preferably, the word frequency statistics subprocess may be specifically configured to:
taking the local list and each other list as list lists to be compared; selecting a search word in the local word frequency data table as a current target, and taking current word frequency data corresponding to the current target in the local word frequency data table as candidate word frequency data; for each list to be compared, if the current target exists in the list to be compared, using the word frequency data corresponding to the current target in the list to be compared as candidate word frequency data; taking the maximum value in the candidate word frequency data corresponding to the current target as the current word frequency data of the current target, performing associative cache to a set intermediate cache table, and returning to execute the selection operation of the current target until search words in the local word frequency data table are all selected; and updating the local word frequency data table based on each search word cached in the middle cache table and the corresponding current word frequency data.
In this embodiment, the word frequency statistics sub-process may perform global property update on the local word frequency data table in combination with the local list and each other list, and the update is performed on each search word in the local word frequency data table. For search terms in the local word frequency data table, the search terms are likely to be located in the list, and in order to ensure the global property of the word frequency data information corresponding to the search terms, a maximum value is selected from the word frequency data in the local list and the word frequency data in the local word frequency data table by combining the word frequency data corresponding to the search terms in the local list and the other list lists. The execution of the word frequency statistics subprocess embodies the traversal of the local word frequency data list, the local list and other list lists.
Further, the list generation sub-process may be specifically configured to:
arranging the search words according to the updated current word frequency data of the search words in the local word frequency data table from large to small; and recording the search words with the ranking sequence numbers smaller than or equal to a set threshold value after arrangement as hot search words, and forming a current list based on the hot search words and corresponding current word frequency data.
In this embodiment, after the execution of the word frequency statistics sub-process ensures the data accuracy of the local word frequency data table, the current list may be further generated by the list generation sub-process. The list can be specifically understood as a list for ranking search terms which are searched relatively frequently, the list generation sub-process can rank the search terms in the local word frequency data table from large to small according to corresponding current word frequency data, then the search terms of K before ranking (namely, the ranking number is less than or equal to K, and K can be set according to an application scene) are intercepted and recorded as hot search terms, and meanwhile, the current list is formed according to the hot search terms and the current word frequency data.
It can be understood that, when the list service process of the provided list service system generates the current list based on the embodiment, the local word frequency data table, the historical local list and other list lists are mainly combined, so that the generated current list avoids the influence of single room errors in the existing system on the overall situation to the maximum extent. Meanwhile, in the running process of the system, a small amount of data is lost, and the overall list result cannot be obviously influenced, so that the effect of effectively achieving data consistency and correctness by using a small amount of data synchronization among multiple nodes is realized.
Example two
Fig. 2 is a flowchart of a list generating method according to a second embodiment of the present invention, where the method is suitable for generating a list, and the method can be executed by the list service system according to the first embodiment of the present invention.
According to the description of the pair of list service systems in the above embodiments, the known list service system specifically includes: at least one listing service process, at least one storage service point, and a search service process.
As shown in fig. 2, the method for generating a list according to the second embodiment specifically includes the following operations:
s201, the list service process acquires a local word frequency data list, a local list and other list lists from the corresponding storage service points.
In this embodiment, the number of the storage service points included in the entire list service system is the same as the number of the machine rooms included in the global service deployment, that is, one storage service point may exist in a machine room corresponding to one region, one machine room may include one or more list service processes created on the server, and the storage service points in the same machine room are associated with the list service processes.
The step specifically provides an operation that the list service process acquires the local word frequency data table, the local list and other list lists from the associated storage service points. It can be understood that the list service processes associated with the same storage service point share a local word frequency data table, a local list and other list lists.
S202, the list service process generates a current list according to the acquired local word frequency data list, the local list and other list lists, and sends the current list to a corresponding storage service point so as to replace the original local list on the storage service point.
For example, in this step, the list service process may combine the local word frequency data table and the corresponding local list lists on all the storage service points (that is, the local list on the associated storage service point and the other list lists where the other storage service points are synchronized to the associated storage service point), may re-determine the current word frequency data for each search term in the local word frequency data table, then may select K search terms with the first K names after being arranged from large to small according to the corresponding current word frequency data, and combine the corresponding current word frequency data to generate the current list. The current list can be sent to the associated storage service point to be used as a new local list to replace the original local list on the corresponding storage service point.
The process of re-determining the current word frequency data of each search word can be expressed as follows: and traversing the local word frequency data table and each list, acquiring the maximum value in the word frequency data corresponding to the search word, and taking the maximum value as the final current word frequency data of the search word.
Compared with the method executed by the existing list service system, the list generation method avoids the influence of single room error data on the whole situation to the maximum extent, and better reflects the data consistency and the correctness of the generated list.
As an optional embodiment provided by the second embodiment of the present invention, the method further optimally includes that the search service process responds to a search request sent by the user terminal, and when it is determined that a search result of a search term associated with the search request is non-empty, the search result is sent to any associated target list service process; and after receiving the search terms, the target list service process determines word frequency increment data of the search terms and feeds the search terms and the corresponding word frequency increment data back to the corresponding storage service points.
In the optional embodiment, the search service process provides word frequency data of search words for the generation of the list, and the whole network synchronization of the word frequency incremental data corresponding to the search words is realized through the running of the target list service process.
As another optional embodiment provided by the second embodiment of the present invention, the method further optimizes a storage service point performed corresponding to the target list service to update a stored local word-frequency data table according to the search term and the corresponding word-frequency incremental data, and synchronizes the search term and the corresponding word-frequency incremental data to other storage service points; and updating the corresponding stored local word frequency data table by each other storage service point according to the search words fed back by the storage service point corresponding to the target list service process and the corresponding word frequency increment data.
On the basis of the optimization, the method further comprises the following steps: when receiving a list display request sent by a user terminal, each list service process feeds back a local list acquired from the corresponding storage service point to the user terminal so as to display the list through the user terminal.
In order to better understand the list generation process of the list service system provided in this embodiment, fig. 3 is a flowchart illustrating an example of implementing the list generation based on the list service system in the second embodiment of the present invention. As shown in fig. 3, a process of interacting among a search service process existing in the list service system, a target list service process, a target storage service point associated with the target list service process, and other storage service points to generate a list is given.
Specifically, as shown in fig. 3, the interaction process of the search service process, the target list service process, the target storage service point associated with the target list service process, and the other storage service points is as follows:
s1, the search service process responds to the search request sent by the user terminal, and sends the search result to the target list service process when the search result of the search word associated with the search request is determined to be non-empty.
In the embodiment, the search service process can send the search terms to any target list service process associated with the search service process, so that the operation that all write traffic of the whole network needs to be concentrated to the main node in the existing system is effectively avoided, and the transverse expansion capability of the service is improved.
S2, the target list service process determines word frequency increment data of the search words and feeds the search words and the corresponding word frequency increment data back to the associated target storage service points.
In this embodiment, the list service process that receives the search service process may be recorded as a target list service process, and the target list service process may count the number of the search terms received at the current time and use the number of the search terms as term frequency increment data. Meanwhile, the target list service process needs to feed back the received search terms and the corresponding word frequency increment data to the storage service points associated with the search terms, and the storage service points are recorded as target storage service points.
And S3, the target storage service point updates the stored local word frequency data table according to the search words and the corresponding word frequency increment data, and synchronizes the search words and the corresponding word frequency increment data to other storage service points.
In this embodiment, the target storage service point performs an operation of updating the local word frequency data table, and simultaneously performs an operation of synchronizing the search word and the corresponding word frequency increment data to other storage service points in order to ensure real-time updating of relevant data on other storage service points.
The step synchronizes the word frequency incremental data, but not the word frequency data, so that the condition that the whole network data is polluted due to the abnormal data storage caused by the abnormal storage caused by a certain storage service point can be avoided.
And S4, updating the corresponding stored local word frequency data table by each other storage service point according to the received search words and the corresponding word frequency increment data.
In this step, other storage service points execute the operation of updating the local word frequency data table. Each storage service point stores and synchronizes the list lists of other storage service points, which is helpful to avoid the inaccuracy of word frequency data caused by the loss of data of part of the storage service points.
S5, the target list service process obtains the local word frequency data list, the local list and other list lists from the target storage service point.
In this step, when the target list service process reaches the condition for generating the list (for example, the time interval between the target list service process and the last operation reaches the set time), the operation of acquiring the local word frequency data table, the local list and each other list from the target storage service point is executed.
And S6, the target list service process generates a current list according to the local word frequency data list, the local list and other list lists.
In the step, the target list service process generates the current list by referring to the global data, so that the data consistency and accuracy of the current list are effectively ensured.
And S7, the target list service process sends the current list to the target storage service point.
In this step, the local list stored in the target storage service point is updated by sending the current list to the target storage service point.
And S8, the target storage service point takes the current list as a new local list to replace the stored local list, and synchronizes the new local list to other storage service points, so that the other storage service points take the new local list to replace the stored other list.
In the step, the updated local list in the target storage service point is synchronized with other storage service points, so that the effective synchronization of the data information of the whole network under the distributed architecture is effectively realized.
EXAMPLE III
Fig. 4 is a schematic diagram of a hardware structure of a server according to a third embodiment of the present invention, specifically, the server is used as a storage service point in the list service system provided in the first embodiment, and/or an execution subject of a list service process in the list service system provided in the first embodiment. The method specifically comprises the following steps: a processor and a storage device. At least one instruction is stored in the storage device, and the instruction is executed by the processor, so that the server can perform the operation steps corresponding to the stored service points and/or the list service process in the list generating method provided by the second embodiment.
Referring to fig. 4, the server may specifically include: a processor 30, a storage device 31, a display 32, an input device 33, an output device 34, and a communication device 35. The number of the processors 30 in the server may be one or more, and one processor 30 is taken as an example in fig. 4. The number of the storage devices 31 in the server may be one or more, and one storage device 31 is taken as an example in fig. 4. The processor 30, the storage device 31, the display 32, the input device 33, the output device 34, and the communication device 35 of the server may be connected by a bus or other means, and fig. 4 illustrates the connection by a bus as an example.
Specifically, in the embodiment, when the processor 30 executes one or more programs stored in the storage device 31, the following is specifically executed, for example: the list service process acquires a local word frequency data list, a local list and other list lists from the corresponding storage service points; and the list service process generates a current list according to the acquired local word frequency data list, the local list and other list lists, and sends the current list to a corresponding storage service point to serve as a new local list.
Embodiments of the present invention also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform the embodiments of the present invention. Illustratively, the foregoing embodiments include: the list service process acquires a local word frequency data list, a local list and other list lists from the corresponding storage service points; and the list service process generates a current list according to the acquired local word frequency data list, the local list and other list lists, and sends the current list to a corresponding storage service point to serve as a new local list.
It should be noted that, for the server and the storage medium embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (16)
1. A list service system, comprising: the system comprises at least one list service process and at least one storage service point, wherein communication connection exists between the storage service points, and the storage service points are in communication connection with the associated list service process;
the storage service point stores a local word frequency data table and a local list related to the associated list service process; other list lists relevant to other storage service points are stored;
and each list service process is used for generating a current list according to the local word frequency data table, the local list and other list lists acquired from the corresponding storage service point, sending the current list to the corresponding storage service point to replace the original local list and storing the current list as a new local list.
2. The system of claim 1, further comprising: the search service process is a process that searches for,
the search service process is used for responding to a search request sent by a user terminal, and sending the search result to any associated target list service process when the search result of the search word associated with the search request is determined to be non-empty.
3. The system of claim 2,
the target list service process is further used for determining word frequency increment data of the search terms after the search terms are received, and feeding back the search terms and the corresponding word frequency increment data to the associated storage service points.
4. The system of claim 3,
the storage service points corresponding to the target list service process are also used for updating the stored local word frequency data table according to the search words and the corresponding word frequency increment data and synchronizing the search words and the corresponding word frequency increment data to other storage service points;
and the other storage service points are used for updating the corresponding stored local word frequency data table according to the received search words and the corresponding word frequency increment data.
5. The system of claim 4, wherein each of the storage service points is specifically configured to:
after receiving the search word and corresponding word frequency increment data, searching whether the search word exists from a stored local word frequency data table;
if the search word exists, determining current word frequency data corresponding to the search word in the local word frequency data table, and storing the sum of the current word frequency data and the word frequency increment data in a correlation mode as new current word frequency data;
and if the search word does not exist, adding the search word to the local word frequency data table, and storing the word frequency increment data serving as the current word frequency data of the search word in the local word frequency data table in a correlation manner.
6. The system of claim 1,
each list service process is further used for feeding back a local list acquired from the corresponding storage service point to the user terminal when receiving a list display request sent by the user terminal, so that list display is performed through the user terminal.
7. The system of claim 1,
the storage service points are used for receiving the current list sent by the corresponding list service process and replacing the stored local list as a new local list, and meanwhile, synchronizing the new local list to other storage service points so that the other storage service points can replace the stored other list lists as new other list lists.
8. The system of any of claims 1-7, wherein each of the listing service processes specifically includes:
the information acquisition sub-process is used for acquiring a local word frequency data table and a local list which are currently stored in the corresponding storage service points and other list lists which are correspondingly fed back by the storage service points;
a word frequency statistics sub-process, configured to update current word frequency data of each search word in the local word frequency data table according to the local list and each other list, so as to obtain a new local word frequency data table;
the list generation sub-process is used for generating a current list according to the updated local word frequency data table;
and the information sending sub-process is used for sending the generated current list to the corresponding storage service point so that the storage service point adopts the current list to replace the stored local list.
9. The system of claim 7, wherein the word frequency statistics subprocess is specifically configured to:
taking the local list and each other list as list lists to be compared;
selecting a search word in the local word frequency data table as a current target, and taking current word frequency data corresponding to the current target in the local word frequency data table as candidate word frequency data;
for each list to be compared, if the current target exists in the list to be compared, using the word frequency data corresponding to the current target in the list to be compared as candidate word frequency data;
taking the maximum value in the candidate word frequency data corresponding to the current target as the current word frequency data of the current target, performing associative cache to a set intermediate cache table, and returning to execute the selection operation of the current target until search words in the local word frequency data table are all selected;
and updating the local word frequency data table based on each search word cached in the middle cache table and the corresponding current word frequency data.
10. The system of claim 7, wherein the list generation sub-process is specifically configured to:
arranging the search words according to the updated current word frequency data of the search words in the local word frequency data table from large to small;
and recording the search words with the ranking sequence numbers smaller than or equal to a set threshold value after arrangement as hot search words, and forming a current list based on the hot search words and corresponding current word frequency data.
11. A list generation method, performed by the list service system recited in any one of claims 1-10, the method comprising:
the list service process acquires a local word frequency data list, a local list and other list lists from the corresponding storage service points;
and the list service process generates a current list according to the acquired local word frequency data list, the local list and other list lists, and sends the current list to the corresponding storage service point so as to replace the original local list on the storage service point.
12. The method of claim 11, further comprising:
the method comprises the steps that a search service process responds to a search request sent by a user terminal, and when the search result of a search word associated with the search request is determined to be non-empty, the search result is sent to any associated target list service process;
and after receiving the search terms, the target list service process determines word frequency increment data of the search terms and feeds the search terms and the corresponding word frequency increment data back to the corresponding storage service points.
13. The method of claim 12, further comprising:
the storage service points which are carried out corresponding to the target list service update a stored local word frequency data table according to the search words and the corresponding word frequency increment data, and synchronize the search words and the corresponding word frequency increment data to other storage service points;
and updating the corresponding stored local word frequency data table by each other storage service point according to the search words fed back by the storage service point corresponding to the target list service process and the corresponding word frequency increment data.
14. The method according to any one of claims 11-13, further comprising:
when receiving a list display request sent by a user terminal, each list service process feeds back a local list acquired from the corresponding storage service point to the user terminal so as to display the list through the user terminal.
15. A server, for serving as a storage service point in the ranking service system of any one of claims 1 to 10 and/or serving as an execution terminal of a ranking service process in the ranking service system of any one of claims 1 to 10, comprising:
one or more processors;
storage means for storing one or more programs;
the one or more programs are executed by the one or more processors such that the one or more processors are operable to perform the chart generation method of any of claims 11-14.
16. A computer-readable storage medium having stored thereon a computer program, the program, when executed by a processor, implementing the chart generation method of any of claims 11-14.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010820861.XA CN112000885B (en) | 2020-08-14 | 2020-08-14 | List service system, list generation method, server and storage medium |
| PCT/CN2021/108218 WO2022033291A1 (en) | 2020-08-14 | 2021-07-23 | Ranking list service system, ranking list generation method, server, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010820861.XA CN112000885B (en) | 2020-08-14 | 2020-08-14 | List service system, list generation method, server and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112000885A true CN112000885A (en) | 2020-11-27 |
| CN112000885B CN112000885B (en) | 2022-09-30 |
Family
ID=73473216
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010820861.XA Active CN112000885B (en) | 2020-08-14 | 2020-08-14 | List service system, list generation method, server and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112000885B (en) |
| WO (1) | WO2022033291A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022033291A1 (en) * | 2020-08-14 | 2022-02-17 | 百果园技术(新加坡)有限公司 | Ranking list service system, ranking list generation method, server, and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150242434A1 (en) * | 2014-02-21 | 2015-08-27 | Google Inc. | Scalable Current Leaderboard Using Delta-Encoded Updates |
| CN106302829A (en) * | 2016-10-26 | 2017-01-04 | 腾讯音乐娱乐（深圳）有限公司 | A kind of information access method, device and server |
| CN107220287A (en) * | 2017-04-24 | 2017-09-29 | 东软集团股份有限公司 | For the index managing method of log query, device, storage medium and equipment |
| CN107341268A (en) * | 2017-07-25 | 2017-11-10 | 北京奇艺世纪科技有限公司 | A kind of heat searches list sort method and system |
| CN110096637A (en) * | 2019-04-16 | 2019-08-06 | 广州虎牙信息科技有限公司 | Method, apparatus, storage medium and the terminal device that more lists generate |
| CN110377611A (en) * | 2019-07-12 | 2019-10-25 | 北京三快在线科技有限公司 | Integrate the method and device of ranking |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10083236B2 (en) * | 2015-05-29 | 2018-09-25 | Salesforce.Com, Inc. | Database searching using a graph of nodes and edges formed using log node pairs |
| US10482096B2 (en) * | 2017-02-13 | 2019-11-19 | Microsoft Technology Licensing, Llc | Distributed index searching in computing systems |
| CN112000885B (en) * | 2020-08-14 | 2022-09-30 | 广州市百果园信息技术有限公司 | List service system, list generation method, server and storage medium |
-
2020
- 2020-08-14 CN CN202010820861.XA patent/CN112000885B/en active Active
-
2021
- 2021-07-23 WO PCT/CN2021/108218 patent/WO2022033291A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150242434A1 (en) * | 2014-02-21 | 2015-08-27 | Google Inc. | Scalable Current Leaderboard Using Delta-Encoded Updates |
| CN106302829A (en) * | 2016-10-26 | 2017-01-04 | 腾讯音乐娱乐（深圳）有限公司 | A kind of information access method, device and server |
| CN107220287A (en) * | 2017-04-24 | 2017-09-29 | 东软集团股份有限公司 | For the index managing method of log query, device, storage medium and equipment |
| CN107341268A (en) * | 2017-07-25 | 2017-11-10 | 北京奇艺世纪科技有限公司 | A kind of heat searches list sort method and system |
| CN110096637A (en) * | 2019-04-16 | 2019-08-06 | 广州虎牙信息科技有限公司 | Method, apparatus, storage medium and the terminal device that more lists generate |
| CN110377611A (en) * | 2019-07-12 | 2019-10-25 | 北京三快在线科技有限公司 | Integrate the method and device of ranking |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2022033291A1 (en) * | 2020-08-14 | 2022-02-17 | 百果园技术(新加坡)有限公司 | Ranking list service system, ranking list generation method, server, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022033291A1 (en) | 2022-02-17 |
| CN112000885B (en) | 2022-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109729111B (en) | Method, apparatus and computer program product for managing distributed systems | |
| CN103577407A (en) | Query method and query device for distributed database | |
| CN110795503A (en) | Multi-cluster data synchronization method and related device of distributed storage system | |
| EP3817338B1 (en) | Method and apparatus for acquiring rpc member information, electronic device and storage medium | |
| CN104468163A (en) | Method and device for disaster tolerant network construction, and disaster tolerant network | |
| KR20100099319A (en) | Node system, server switching method, server device, and data transfer method | |
| CN107172214B (en) | Service node discovery method and device with load balancing function | |
| CN107370809A (en) | Method of data synchronization and data search system | |
| CN112000885B (en) | List service system, list generation method, server and storage medium | |
| CN111526188B (en) | System and method for ensuring zero data loss based on Spark Streaming in combination with Kafka | |
| US10091288B2 (en) | Ordered execution of tasks | |
| EP3427157B1 (en) | Cross-regional data transmission | |
| CN114500257A (en) | Network configuration distribution method, device, control node and storage medium | |
| CN113259476A (en) | Message pushing method and system | |
| US9870402B2 (en) | Distributed storage device, storage node, data providing method, and medium | |
| CN112000850A (en) | Method, device, system and equipment for data processing | |
| CN110083653A (en) | A kind of operating method of order data, device, computer equipment and storage medium | |
| CN114679349B (en) | Data communication method and device | |
| CN109522294A (en) | A kind of distributed data cache system and data cache method | |
| CN111600958B (en) | Service discovery system, service data management method, server, and storage medium | |
| CN109257403A (en) | Date storage method and equipment, distributed memory system | |
| JP2013171483A (en) | Difference replication system, master database device and slave database device | |
| CN113037797A (en) | Data processing method and device | |
| CN113157604B (en) | Data acquisition method and device based on distributed system and related products | |
| CN115168366B (en) | Data processing method, data processing device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20231008 Address after: 31a, 15th floor, building 30, maple commercial city, bangrang Road, Brazil Patentee after: Baiguoyuan Technology (Singapore) Co.,Ltd. Address before: 511402 5-13 / F, West Tower, building C, 274 Xingtai Road, Shiqiao street, Panyu District, Guangzhou City, Guangdong Province Patentee before: GUANGZHOU BAIGUOYUAN INFORMATION TECHNOLOGY Co.,Ltd. |
|
| TR01 | Transfer of patent right |