CN114003571A - Data processing method and device, electronic equipment and storage medium - Google Patents
Data processing method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN114003571A CN114003571A CN202111277347.7A CN202111277347A CN114003571A CN 114003571 A CN114003571 A CN 114003571A CN 202111277347 A CN202111277347 A CN 202111277347A CN 114003571 A CN114003571 A CN 114003571A
- Authority
- CN
- China
- Prior art keywords
- account
- list
- data
- target
- request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 15
- 238000000034 method Methods 0.000 claims abstract description 45
- 238000012545 processing Methods 0.000 claims abstract description 24
- 238000012986 modification Methods 0.000 claims description 27
- 230000004048 modification Effects 0.000 claims description 27
- 238000012544 monitoring process Methods 0.000 claims description 9
- 238000013475 authorization Methods 0.000 claims description 6
- 238000005192 partition Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 abstract description 16
- 238000012550 audit Methods 0.000 description 30
- 238000010586 diagram Methods 0.000 description 8
- 238000012546 transfer Methods 0.000 description 3
- 229920004934 Dacron® Polymers 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000005020 polyethylene terephthalate Substances 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 230000004931 aggregating effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 230000003442 weekly effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/17—Details of further file system functions
- G06F16/172—Caching, prefetching or hoarding of files
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/11—File system administration, e.g. details of archiving or snapshots
- G06F16/128—Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/252—Integrating or interfacing systems involving database management systems between a Database Management System and a front-end application
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Transfer Between Computers (AREA)
Abstract
The disclosure relates to a data processing method, a data processing device, a server and a storage medium, wherein the method comprises the steps of obtaining account hotspot characteristic information, calculating the heat value of a target account according to the account hotspot characteristic information, generating an account list according to the heat value, and storing the account list in a cache, so that when an account list request is received, the account list can be directly obtained from the cache. Because the related account hotspot characteristic information used for calculating the heat value is directly acquired from the outside, the calculation pressure of directly calculating the account hotspot characteristic information through the server can be reduced, and the pressure of high concurrent data requests can be relieved in a mode of caching the account list, so the performance of the server can be improved.
Description
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to a data processing method and apparatus, an electronic device, and a storage medium.
Background
With the development of internet technology, the interaction of users is in diverse changes. At present, the hit leaderboards in the same city business are ranked according to the related data of the hit heat value, and the calculation of the related data of the hit heat value consumes excessive resources of the server. And under the highly concurrent leader board request task of the user, the QPS (Queries-per-second) of the server is caused to be too high.
Therefore, in the current city business, the server has the problem of slow response speed due to resource consumption and too high QPS.
Disclosure of Invention
The present disclosure provides a data processing method, an apparatus, an electronic device, and a storage medium, so as to at least solve the problem in the related art that a server has a slow response speed due to resource consumption and an excessively high QPS. The technical scheme of the disclosure is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided a data processing method, including
Acquiring account hotspot characteristic information, wherein the account hotspot characteristic information is acquired from data authorized by a target account;
calculating the heat value of the target account according to the account hotspot characteristic information, generating an account list according to the heat value, and storing the account list in a cache;
and when an account list request is received, acquiring the account list from the cache.
In one embodiment, the data authorized by the target account includes location information of the target account; the method further comprises the following steps: monitoring the modification operation of the location information of the homepage of the target account based on a preset period; when the situation that the location information of the homepage of the target account is modified is monitored, the modification information of the location of the homepage of the target account is obtained, and the location information of the target account in the data authorized by the target account is updated according to the modification information.
In one embodiment, the method further comprises: when the update time of the account list is up, calculating the hot value of the target account according to the account hot feature information, updating the account list according to the hot value, and updating the cache according to the updated account list, wherein the account hot feature information is the hot feature information acquired from the updated target account authorization data.
In one embodiment, the storing the account list in a cache includes: creating a snapshot of the account manifest, the snapshot including a storage file in at least one storage partition for storing the account manifest; and storing the snapshot of the account list in a cache.
In one embodiment, after the calculating the hot value of the target account according to the account hot spot feature information, the method further includes: and sending the hot value of the target account to a data center station, wherein the hot value is used for indicating the data center station to store the hot value of the target account.
In one embodiment, the method further comprises: when the updating time of the account list is up, calculating the heat value of the target account according to the account hot feature information, updating the account list according to the heat value, and updating the cache according to the updated account list, wherein the account hot feature information is acquired according to the updated data authorized by the target account.
In one embodiment, the method further comprises: receiving a service request of a user account, and acquiring a request state of the user account; and when the service request is determined to be valid according to the request state of the user account, auditing the service request and updating the request state of the user account.
In one embodiment, after the auditing the service request, the method further includes: and receiving an audit result of auditing the service request, returning the audit result to the user account, and updating the request state of the user account according to the audit result.
In one embodiment, the method further comprises: and when the service request is determined to be invalid according to the request state of the user account, returning prompt information that the service request is invalid to the user account.
In one embodiment, after obtaining the account hotspot characteristic information sent by the data platform, the method further includes: and locally storing the account hotspot characteristic information.
According to a second aspect of the embodiments of the present disclosure, there is provided a data processing apparatus including:
the account hotspot characteristic information acquisition module is configured to execute account hotspot characteristic information acquisition, wherein the account hotspot characteristic information is hotspot characteristic information acquired from data authorized by a target account;
the account list generating module is configured to calculate the heat value of the target account according to the account hotspot characteristic information and generate an account list according to the heat value;
an account manifest caching module configured to perform storing the account manifest into a cache;
a request processing module configured to perform, when an account manifest request is received, acquiring the account manifest from the cache.
In one embodiment, the data authorized by the target account includes location information of the target account; the device further comprises: the modification monitoring module is configured to execute modification operation for monitoring location information of the target account homepage based on a preset period; and the data updating module is configured to acquire modification information of the location of the home page of the target account when the fact that the information of the location of the home page of the target account is modified is monitored, and update the location information of the target account in the data authorized by the target account according to the modification information.
In one embodiment, the apparatus further comprises: and the account list updating module is configured to calculate a hot value of the target account according to the account hot feature information when the account list updating time is up, update the account list according to the hot value, and update the cache according to the updated account list, wherein the account hot feature information is hot feature information acquired from the updated target account authorization data.
In one embodiment, the account manifest caching module is further configured to perform: creating a snapshot of the account manifest, the snapshot including a storage file in at least one storage partition for storing the account manifest; and storing the snapshot of the account list in a cache.
In one embodiment, the apparatus further comprises: a hot value sending module configured to execute sending the hot value of the target account to a data center station, where the hot value is used to instruct the data center station to store the hot value of the target account.
In one embodiment, the apparatus further comprises: the service request receiving module is configured to execute receiving of a service request of a user account and obtain a request state of the user account; and the service request processing module is configured to perform auditing on the service request and update the request state of the user account when the service request is determined to be valid according to the request state of the user account.
In one embodiment, the apparatus further comprises: and the audit result feedback module is configured to receive an audit result of auditing the service request, return the audit result to the user account, and update the request state of the user account according to the audit result.
In one embodiment, the service request processing module is further configured to perform: and when the service request is determined to be invalid according to the request state of the user account, returning prompt information that the service request is invalid to the user account.
In one embodiment, the apparatus further comprises: and the account hotspot characteristic information storage module is configured to locally store the account hotspot characteristic information.
According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including: a processor; a memory for storing the processor-executable instructions; wherein the processor is configured to execute the instructions to implement the data processing method of the first aspect as described above.
According to a fourth aspect of embodiments of the present disclosure, there is provided a computer-readable storage medium, in which instructions, when executed by a processor of an electronic device, enable the electronic device to perform the data processing method of the first aspect.
According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program product, which includes instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the data processing method of the first aspect.
The technical scheme provided by the embodiment of the disclosure at least brings the following beneficial effects: by acquiring account hotspot characteristic information, calculating the heat value of the target account according to the account hotspot characteristic information, generating an account list according to the heat value, and storing the account list in a cache, when an account list request is received, the account list can be directly acquired from the cache. Because the related account hotspot characteristic information used for calculating the heat value is directly acquired from the outside, the calculation pressure of directly calculating the account hotspot characteristic information through the server can be reduced, and the pressure of high concurrent data requests can be relieved in a mode of caching the account list, so the performance of the server can be improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure and are not to be construed as limiting the disclosure.
FIG. 1 is a diagram illustrating an application environment for a method of data processing, according to an exemplary embodiment.
FIG. 2 is a flow chart illustrating a method of data processing according to an exemplary embodiment.
Fig. 3 is a flow chart illustrating a method of data processing according to another exemplary embodiment.
FIG. 4 is a diagram illustrating a data processing application, according to an example embodiment.
FIG. 5 is a diagram illustrating an update flow of a peer list according to an example embodiment.
FIG. 6 is a schematic flow diagram illustrating a Daidan application according to an exemplary embodiment.
FIG. 7 is a block diagram illustrating a data processing apparatus according to an example embodiment.
FIG. 8 is a block diagram illustrating an electronic device in accordance with an example embodiment.
Detailed Description
In order to make the technical solutions of the present disclosure better understood by those of ordinary skill in the art, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein are capable of operation in sequences other than those illustrated or otherwise described herein. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for presentation, analyzed data, etc.) referred to in the present disclosure are both information and data that are authorized by the user or sufficiently authorized by various parties.
The data processing method provided by the present disclosure can be applied to the application environment shown in fig. 1. Wherein, the server 110 interacts with the data platform 120 and the terminal 130 respectively through the network. Specifically, the server 110 obtains account hotspot feature information sent by the data platform 120, calculates the heat value of the target account according to the account hotspot feature information, further generates an account list according to the heat value, stores the account list in a cache, and obtains the account list from the cache and returns the account list to the terminal 130 when receiving an account list request sent by the terminal 130. Because the related account hotspot characteristic information used for calculating the heat value is directly acquired from the data platform, the calculation pressure of directly calculating the account hotspot characteristic information through the server can be reduced, and the pressure of high concurrent data requests can be relieved in a mode of caching the account list, so the performance of the server can be improved.
In particular, the server 110 may be implemented as a stand-alone server or a server cluster composed of a plurality of servers. The data platform 120 is an environment or condition for aggregating data collection, storage, operations and result calls, such as an operating environment of computer hardware or software. The terminal 130 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices.
Fig. 2 is a flowchart illustrating a data processing method according to an exemplary embodiment, and as shown in fig. 2, the method includes the following steps, for example, when applied to the server in fig. 1.
In step S210, account hotspot characteristic information is acquired.
The account hotspot characteristic information is obtained by obtaining target account authorization data by external other equipment such as a data platform and obtaining the target account authorization data. Specifically, the data authorized by the target account may be user information of the account stored in the database, and the user information includes, but is not limited to, an account identifier of the user, an attribute identifier of the user, work data posted by the user, a situation in which the work data is played, a situation in which the user is concerned, and a situation in which the user approves. The attribute identifier of the user may be attribute information of the user, for example, an account that has become a visitor has a corresponding visitor attribute, an account with different preferences has a corresponding preference attribute, or an account with a corresponding age attribute according to the age of the user.
The account hotspot characteristic information is account related information which is obtained by performing statistical calculation on data authorized by the target account based on a specific application scene and can reflect corresponding scene characteristics, and it can be understood that the account hotspot characteristic is a scene characteristic which has a certain influence range and is highly concerned in the range. For example, in the case of a hit scene, there is usually a hit list in the same city, and the factors that depend on the hit list can be referred to as corresponding account hotspot characteristic information. Similarly, if a cate arrival list is to be generated, the factor on which the cate arrival list is generated may also be referred to as corresponding account hotspot characteristic information. Specifically, for example, taking the generation of the same-city leader list as an example, the corresponding account hotspot characteristic information includes, but is not limited to, fan amount of the target account, played amount of the published work, attention amount, praise amount, and the like, which can be obtained by performing statistical calculation on data authorized by the target account. It can be understood that, in different application scenarios, the corresponding account hotspot characteristic information may be different.
In this embodiment, taking the external other device that performs statistical calculation on the data authorized by the target account to obtain account hotspot characteristic information as the data platform, the data platform may obtain the corresponding account hotspot characteristic information by obtaining the data authorized by the target account in the database and performing statistical calculation on the data authorized by the target account, so as to send the account hotspot characteristic information to the server. Therefore, the server can receive the account hotspot characteristic information sent by the data platform, and further generate a corresponding list through subsequent steps and perform list processing.
In step S220, the heat value of the target account is calculated according to the account hotspot characteristic information, an account list is generated according to the heat value, and the account list is stored in the cache.
The hot value is obtained by performing dimension processing on account hot spot characteristic information under the condition of comprehensively considering the account hot spot characteristic information. For example, the heat value may be obtained by weighting a plurality of pieces of information in the account hotspot characteristic information. The account list is an ordered list obtained by sequencing each target account according to the popularity value, and is like a city darby list, a food darby list and the like. In this embodiment, the server may calculate a hot value of the target account according to the account hotspot characteristic information, sort the target account according to the hot value to generate an account list, and store the account list in the cache.
In step S230, when the account list request is received, the account list is acquired from the cache.
The account list request is an indication or command for requesting an account list from the server. In this embodiment, when the server receives the account list request, the server obtains the account list from the cache and returns the account list. Since the cache is a storage for high-speed data exchange and performs data exchange before the memory, the cache has a faster rate. Therefore, the account list is stored in the cache, and the account list can be directly obtained from the cache when needed, so that interaction with a memory is not needed, the pressure of data request can be relieved, and the request feedback rate can be improved.
In the data processing method, the account hotspot characteristic information is acquired, the heat value of the target account is calculated according to the account hotspot characteristic information, the account list is generated according to the heat value, and the account list is stored in the cache, so that the account list can be directly acquired from the cache when an account list request is received. Because the related account hotspot characteristic information used for calculating the heat value is directly acquired from the data platform, the calculation pressure of directly calculating the account hotspot characteristic information through the server can be reduced, and the pressure of high concurrent data requests can be relieved in a mode of caching the account list, so the performance of the server can be improved.
In an exemplary embodiment, the data authorized by the target account includes location information of the target account; the method may further include: and monitoring the modification operation of the location information of the homepage of the target account based on a preset period, acquiring the modification information of the location of the homepage of the target account when the modification of the location information of the homepage of the target account is monitored, and updating the location information of the target account in the data authorized by the target account according to the modification information. The preset period refers to a preset time interval between two adjacent monitoring, specifically, the preset period may be preset X seconds or N minutes, and when the preset period is shorter, the real-time monitoring is equivalent. In this embodiment, for example, in the same-city visitor scenario, in order to ensure that the visitor can appear in the list of the corresponding city after modifying the location information of the corresponding account, the present embodiment monitors the modification operation of the location information of the home page of the target account based on a preset period, that is, monitors the modification operation of the location information of the visitor account, and obtains corresponding modification information when the modification operation is monitored, so as to update the location information of the target account in the data authorized by the target account according to the modification information corresponding to the monitored modification operation. Therefore, the target account in the subsequent list calculation can appear in the modified city list corresponding to the location. The embodiment monitors the key data in the service, thereby realizing real-time updating of the data and having higher real-time performance.
In an exemplary embodiment, the method may further include: when the updating time of the account list is up, calculating the heat value of the target account according to the account hot feature information, updating the account list according to the heat value, and updating the cache according to the updated account list, wherein the account hot feature information is acquired according to the updated authorized data of the target account. The account list updating time can be configured in advance, for example, daily timing updating, weekly timing updating and the like can be configured in advance. Therefore, when the pre-configured account list updating time is reached, the corresponding account hotspot characteristic information can be obtained based on the updated authorized data of the target account, the heat value of the target account is calculated according to the account hotspot characteristic information, the account list is updated according to the heat value, and the cache is updated according to the updated account list. According to the method and the device, the account list is updated by adopting the updated authorized data of the target account, so that the account list has higher accuracy, and the system loss caused by frequent updating of the system can be reduced by updating the account list in a mode of setting the updating time.
In an exemplary embodiment, the storing the account list in the cache may specifically further include: and creating a snapshot of the account list, and storing the snapshot of the account list in a cache. The snapshot includes a storage file in at least one storage partition for storing the account list, and the corresponding relation between the logical address and the physical address of the storage data is usually recorded in the storage file corresponding to the snapshot, so that the space required by the snapshot is smaller than that required by directly storing the account list, and the performance of the server can be further improved.
In an exemplary embodiment, after calculating the hot value of the target account according to the account hot spot feature information, the method may further include: and sending the hot value of the target account to the data center station, so that the data center station can store the hot value of the target account to provide data support for subsequent various statistical analyses. The data center station is generated for meeting the fast iteration requirement of the foreground and the stability requirement of the background relative to a foreground client and a background server, and the core of the data center station is that the current station requirement comes temporarily, the center station can respond fast, so that the research and development efficiency is improved, the innovation cost is reduced, the working efficiency is improved, the hot value of a target account calculated by the data center station is stored, and the storage pressure of the server is reduced.
In an exemplary embodiment, as shown in fig. 3, the method may further include:
in step S310, a service request of a user account is received, and a request status of the user account is obtained.
The service request refers to an identity request, an attribute request and the like of an account. For example, a request to become a grandman account, a request to become a large V account, etc. The request state includes but is not limited to an unsolicited state, a state under audit, a state that the audit is passed, a state that the audit is not passed, and the like. Specifically, the non-application state indicates that the user account has not initiated a corresponding service request before; the state in the audit indicates that the user account initiates a corresponding service request once, and the service request is in the audit; the state of the audit is passed, which indicates that the user account has initiated a corresponding service request and the service request has passed the audit; and the state that the audit is not passed indicates that the user account initiates the corresponding service request once, but the service request is not passed after the audit.
In this embodiment, taking the service request as the user account request as an example, the user account may initiate a service request to become the user account to the server through the terminal, and after receiving the corresponding service request, the server queries and obtains a request status of the user account, and further processes the service request of the user account through subsequent steps based on the request status of the user account.
In step S320, when the service request is determined to be valid according to the request status of the user account, the service request is checked, and the request status of the user account is updated.
In step S330, when the service request is determined to be invalid according to the request status of the user account, a prompt message indicating that the service request is invalid is returned to the user account.
The service request effectively represents that the service request is legal, so the service request can be accepted and further processed. And the invalid service request indicates that the service request is illegal and thus cannot be accepted. In this embodiment, whether the service request is valid may be determined according to the request status of the user account initiating the service request. For example, if the request state of the user account is a state in the audit, it indicates that the user account has initiated a corresponding service request and the service request is in the audit, so that to avoid the user repeatedly applying, it may be determined that the current service request of the user account is invalid, and thus prompt information that the service request is invalid is returned to the user account. Similarly, when the request status of the user account is the status of passing the audit, the current service request of the user account is also invalid.
For another example, if the request state of the user account is an unapplied state, it indicates that the user account has not initiated a corresponding service request before, that is, the service request of this time is the first request of the user account, and therefore, it can be determined that the current service request of the user account is valid, so that the service request can be audited, and the request state of the user account can be updated at the same time, that is, the unapplied state is updated to the auditing state, so that the request state of the user account is updated in real time, and repeated applications of the user account are avoided. When the request state of the user account is the state of not passing the audit, the reason of not passing the audit can be further obtained, whether the user account currently meets the application condition or not is judged, if the user account currently meets the application condition, the current service request of the user account can be determined to be valid, so that the service request can be audited, and meanwhile, the request state of the user account is updated, namely, the state of not passing the audit is updated to be the state in the audit; and when the user account does not meet the application condition currently, determining that the current service request of the user account is invalid, and returning prompt information that the service request is invalid to the user account.
In the embodiment, the request state of the user account is acquired by receiving the service request of the user account, and when the service request is determined to be valid according to the request state of the user account, the service request is checked and the request state of the user account is updated, so that the request state of the user account is updated in real time; and when the service request is determined to be invalid according to the request state of the user account, returning prompt information that the service request is invalid to the user account so as to avoid repeated application of the user account.
In an exemplary embodiment, after auditing the service request, the method may further include: and receiving an audit result of auditing the service request, returning the audit result to the user account, and updating the request state of the user account according to the audit result. And the auditing result comprises an auditing passing result and an auditing failing result. Specifically, when the received auditing result of the service request is that the auditing is passed, the auditing passed result is returned to the corresponding user account, and the request state of the user account is updated to be the auditing passed state according to the auditing passed result. And when the received auditing result of the service request for auditing is that the auditing is not passed, returning the result of the auditing to the corresponding user account, and updating the request state of the user account to be the state of the auditing being not passed according to the result of the auditing being not passed. Therefore, the request state of the user account can be updated in real time.
In an exemplary embodiment, after obtaining the account hotspot characteristic information sent by the data platform, the method may further include: the corresponding account hotspot characteristic information is locally stored, so that the account hotspot characteristic information can be backed up, the situation that data cannot be acquired due to the failure of a data platform is avoided, and the method can be used for other services, such as statistical services or other services based on a certain requirement.
In an exemplary embodiment, as shown in fig. 4, the following further illustrates the solution of the present application by taking the hotspot data as a city list as an example:
specifically, the data platform transfers and stores a user information table (namely data authorized by a target account, representing user information metadata which has become a person who arrives) in a database to a hive table 1 of the data platform through a mysql2hive task (data transfer tool), then calculates account hotspot characteristic information of the person who arrives according to the hive table 1 through a calculation task, namely the account hotspot characteristic information includes but is not limited to the number of newly added fans of the person who arrives, the playing amount of the city-owned works, the amount of praise and the like, and further transmits a calculation result to a city-owned service system (namely a server) through a message queue by using the hive2kafka task (data transfer tool) so as to be used by the server for calculating the heat value of the person who arrives.
After receiving the account hotspot characteristic information sent by the data platform, the server starts the kafka consumer to consume the account hotspot characteristic information of the dacron, calculates the heat value of the dacron according to a calculation formula, and then stores the heat value in a list counting middle station (namely a data middle station). And after generating a list (namely an account list) according to the heat value, generating a snapshot by day and storing the snapshot into a cache (reloadable cache). When the server receives a list request of a user (a request initiated by the user through a client), the list request is processed through an Application Programming Interface (API), and the list is taken out from the buffer and returned to the corresponding user, so that the problem of high QPS (quick request to setup) of the list request is solved. In addition, the server monitors binlog (log) of the information modification of the personal home page location of the user in real time through the kbus component, and updates the city to which the user information of the user information table belongs in the mysql when the information modification of the personal home page location of the user is monitored, so that the situation that the user can appear in the list of the corresponding city after the location of the user is modified is ensured.
In addition, information about the request status of the same city owner, the city to which the owner belongs, etc. may be stored in redis (a data structure server stored in memory). And for the application auditing operation of the user, updating the redis cache in real time based on the auditing result, and displaying the in-person auditing state of the in-person application page of the client in real time.
For the client, the corresponding works are verified through a redis cache after the reason label is issued by the same owner, and the redis return source is realized through a kbus component, so that the active return source is not made, and the problem of high QPS of the video feed stream is solved.
In an exemplary embodiment, as shown in fig. 5, the update process of the peer list is further described, which specifically includes:
the data platform initiates the list calculation process periodically, for example, the calculation process may be initiated at 0 o' clock each day. Specifically, when the computing process is detected to be started, the user information table in the mysql is transferred to the hive table 1 of the data platform through the mysql2hive task, then the account hotspot feature information of the user is computed through the computing task according to the hive table 1, and then the hive2kafka task is used for transmitting the computing result to the server through the message queue. And the server starts the account hotspot characteristic information of the kafka connurer consumption chief, namely, the popularity value of the chief is calculated according to a calculation formula, and the list in the middle station of the list is updated according to the popularity value. And meanwhile, the account hotspot characteristic information calculated by the data platform is backed up and stored. In addition, the list in the cache is refreshed through the timed task, and when the timed task is started, the latest list data is requested from the list middle station, so that the local cache is updated. Therefore, the cached list has high instantaneity.
In an exemplary embodiment, as shown in fig. 6, the process of the filed by dawner is further described, which specifically includes:
the user enters a visitor application page through the client or h5 to initiate a visitor application, the server requests page configuration from the API based on application triggering of the user, the API queries the visitor state of the current user from the redis cache, judges whether the application of the current user is valid or not after the visitor state of the current user is obtained, and returns prompt information which cannot be applied to the user if the application of the user is determined to be invalid. If the user's application is determined to be valid, the status of the application conditions is queried, for example, which conditions the applicant needs to have and whether the user currently meets the conditions. And returning the state of the application condition to the user, and if some conditions of the user are not met, the user can finish the condition or quit the application based on the state of the application condition. If the user meets all conditions, the application can be further improved, if the user pays attention to the account of the same city daemon and submits the application to the API, the API writes the corresponding application into the user information table, further receives the writing result and returns the writing result to the user. Meanwhile, the user arrival state in the redis cache is updated based on the writing result, so that the user arrival state is updated in real time.
It should be understood that although the various steps in the flowcharts of fig. 1-6 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 1-6 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed in turn or alternately with other steps or at least some of the other steps.
It is understood that the same/similar parts between the embodiments of the method described above in this specification can be referred to each other, and each embodiment focuses on the differences from the other embodiments, and it is sufficient that the relevant points are referred to the descriptions of the other method embodiments.
FIG. 7 is a block diagram illustrating a data processing apparatus according to an example embodiment. Referring to fig. 7, the apparatus includes an account hotspot characteristic information acquiring module 702, an account list generating module 704, an account list caching module 706 and a request processing module 708.
An account hotspot characteristic information obtaining module 702, configured to perform obtaining of account hotspot characteristic information, where the account hotspot characteristic information is obtained from data authorized by a target account;
the account list generating module 704 is configured to calculate a heat value of the target account according to the account hotspot characteristic information, and generate an account list according to the heat value;
an account manifest caching module 706 configured to perform storing the account manifest into a cache;
a request processing module 708 configured to perform, upon receiving an account manifest request, retrieving the account manifest from the cache.
In an exemplary embodiment, the data authorized by the target account includes location information of the target account; the device further comprises: the modification monitoring module is configured to execute modification operation for monitoring location information of the target account homepage based on a preset period; and the data updating module is configured to acquire modification information of the location of the home page of the target account when the fact that the information of the location of the home page of the target account is modified is monitored, and update the location information of the target account in the data authorized by the target account according to the modification information.
In an exemplary embodiment, the apparatus further comprises: and the account list updating module is configured to calculate the heat value of the target account according to the account hot feature information when the account list updating time is up, update the account list according to the heat value, and update the cache according to the updated account list, wherein the account hot feature information is acquired according to the updated authorized data of the target account.
In an exemplary embodiment, the account manifest caching module is further configured to perform: creating a snapshot of the account manifest, the snapshot including a storage file in at least one storage partition for storing the account manifest; and storing the snapshot of the account list in a cache.
In an exemplary embodiment, the apparatus further comprises: a hot value sending module configured to execute sending the hot value of the target account to a data center station, where the hot value is used to instruct the data center station to store the hot value of the target account.
In an exemplary embodiment, the apparatus further comprises: the service request receiving module is configured to execute receiving of a service request of a user account and obtain a request state of the user account; and the service request processing module is configured to perform auditing on the service request and update the request state of the user account when the service request is determined to be valid according to the request state of the user account.
In an exemplary embodiment, the apparatus further comprises: and the audit result feedback module is configured to receive an audit result of auditing the service request, return the audit result to the user account, and update the request state of the user account according to the audit result.
In an exemplary embodiment, the service request processing module is further configured to perform: and when the service request is determined to be invalid according to the request state of the user account, returning prompt information that the service request is invalid to the user account.
In an exemplary embodiment, the apparatus further comprises: and the account hotspot characteristic information storage module is configured to locally store the account hotspot characteristic information.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
FIG. 8 is a block diagram illustrating an electronic device for data processing S00, according to an example embodiment. For example, the electronic device S00 may be a server. Referring to FIG. 8, electronic device S00 includes a processing component S20 that further includes one or more processors and memory resources represented by memory S22 for storing instructions, such as applications, that are executable by processing component S20. The application program stored in the memory S22 may include one or more modules each corresponding to a set of instructions. Further, the processing component S20 is configured to execute instructions to perform the above-described method.
The electronic device S00 may further include: the power supply module S24 is configured to perform power management of the electronic device S00, the wired or wireless network interface S26 is configured to connect the electronic device S00 to a network, and the input/output (I/O) interface S28. The electronic device S00 may operate based on an operating system stored in the memory S22, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or the like.
In an exemplary embodiment, a computer-readable storage medium comprising instructions, such as the memory S22 comprising instructions, executable by the processor of the electronic device S00 to perform the above method is also provided. The storage medium may be a computer-readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
In an exemplary embodiment, there is also provided a computer program product comprising instructions executable by a processor of the electronic device S00 to perform the above method.
It should be noted that the descriptions of the above-mentioned apparatus, the electronic device, the computer-readable storage medium, the computer program product, and the like according to the method embodiments may also include other embodiments, and specific implementations may refer to the descriptions of the related method embodiments, which are not described in detail herein.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. A method of data processing, the method comprising:
acquiring account hotspot characteristic information, wherein the account hotspot characteristic information is acquired from data authorized by a target account;
calculating the heat value of the target account according to the account hotspot characteristic information, generating an account list according to the heat value, and storing the account list in a cache;
and when an account list request is received, acquiring the account list from the cache.
2. The method of claim 1, wherein the data of the target account authorization includes location information of the target account; the method further comprises the following steps:
monitoring the modification operation of the location information of the homepage of the target account based on a preset period;
when the situation that the location information of the homepage of the target account is modified is monitored, the modification information of the location of the homepage of the target account is obtained, and the location information of the target account in the data authorized by the target account is updated according to the modification information.
3. The method of claim 2, further comprising:
when the update time of the account list is up, calculating the hot value of the target account according to the account hot feature information, updating the account list according to the hot value, and updating the cache according to the updated account list, wherein the account hot feature information is the hot feature information acquired from the updated target account authorization data.
4. The method of claim 1, wherein storing the account manifest in a cache comprises:
creating a snapshot of the account manifest, the snapshot including a storage file in at least one storage partition for storing the account manifest;
and storing the snapshot of the account list in a cache.
5. The method of claim 1, wherein after calculating the hot value of the target account according to the account hot spot signature information, the method further comprises:
and sending the hot value of the target account to a data center station, wherein the hot value is used for indicating the data center station to store the hot value of the target account.
6. The method of claim 1, further comprising:
receiving a service request of a user account, and acquiring a request state of the user account;
and when the service request is determined to be valid according to the request state of the user account, auditing the service request and updating the request state of the user account.
7. A data processing apparatus, comprising:
the account hotspot characteristic information acquisition module is configured to execute account hotspot characteristic information acquisition, wherein the account hotspot characteristic information is hotspot characteristic information acquired from data authorized by a target account;
the account list generating module is configured to calculate the heat value of the target account according to the account hotspot characteristic information and generate an account list according to the heat value;
an account manifest caching module configured to perform storing the account manifest into a cache;
a request processing module configured to perform, when an account manifest request is received, acquiring the account manifest from the cache.
8. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the data processing method of any one of claims 1 to 6.
9. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the data processing method of any of claims 1 to 6.
10. A computer program product comprising instructions which, when executed by a processor of an electronic device, enable the electronic device to perform the data processing method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111277347.7A CN114003571A (en) | 2021-10-29 | 2021-10-29 | Data processing method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111277347.7A CN114003571A (en) | 2021-10-29 | 2021-10-29 | Data processing method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114003571A true CN114003571A (en) | 2022-02-01 |
Family
ID=79925711
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111277347.7A Pending CN114003571A (en) | 2021-10-29 | 2021-10-29 | Data processing method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114003571A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114723558A (en) * | 2022-02-23 | 2022-07-08 | 盈立数智科技(深圳)有限公司 | Method and system for generating quotation list |
| CN115129988A (en) * | 2022-06-29 | 2022-09-30 | 北京达佳互联信息技术有限公司 | Information acquisition method and device, electronic equipment and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108495195A (en) * | 2018-03-13 | 2018-09-04 | 武汉斗鱼网络科技有限公司 | A kind of network direct broadcasting ranking list generation method, device, equipment and storage medium |
| CN110807536A (en) * | 2019-09-30 | 2020-02-18 | 北京三快在线科技有限公司 | Account information association method and device, electronic equipment and storage medium |
| CN112035046A (en) * | 2020-09-10 | 2020-12-04 | 脸萌有限公司 | List information display method and device, electronic equipment and storage medium |
| CN112383785A (en) * | 2020-10-12 | 2021-02-19 | 广州市百果园网络科技有限公司 | Method and device for processing heat value and electronic equipment |
-
2021
- 2021-10-29 CN CN202111277347.7A patent/CN114003571A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108495195A (en) * | 2018-03-13 | 2018-09-04 | 武汉斗鱼网络科技有限公司 | A kind of network direct broadcasting ranking list generation method, device, equipment and storage medium |
| CN110807536A (en) * | 2019-09-30 | 2020-02-18 | 北京三快在线科技有限公司 | Account information association method and device, electronic equipment and storage medium |
| CN112035046A (en) * | 2020-09-10 | 2020-12-04 | 脸萌有限公司 | List information display method and device, electronic equipment and storage medium |
| CN112383785A (en) * | 2020-10-12 | 2021-02-19 | 广州市百果园网络科技有限公司 | Method and device for processing heat value and electronic equipment |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114723558A (en) * | 2022-02-23 | 2022-07-08 | 盈立数智科技(深圳)有限公司 | Method and system for generating quotation list |
| CN115129988A (en) * | 2022-06-29 | 2022-09-30 | 北京达佳互联信息技术有限公司 | Information acquisition method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111464615B (en) | Request processing method, device, server and storage medium | |
| CN114003571A (en) | Data processing method and device, electronic equipment and storage medium | |
| CN102449602A (en) | System and method for reducing startup cost of a software application | |
| US11615443B2 (en) | System and method to selectively update supplemental content rendered in placement regions of a rendered page | |
| CN107423404B (en) | Flow instance data synchronous processing method and device | |
| CN103765387A (en) | Distributed resource management in a portable computing device | |
| CN114006946B (en) | Method, device, equipment and storage medium for processing homogeneous resource request | |
| CN113301390B (en) | Data processing method, device and server for calling virtual resources | |
| CN111108482B (en) | Optimizing network utilization | |
| CN113476853A (en) | Data processing method and device for interactive tasks, electronic equipment and storage medium | |
| CN116028696A (en) | Resource information acquisition method and device, electronic equipment and storage medium | |
| CN112866339A (en) | Data transmission method and device, computer equipment and storage medium | |
| Jia et al. | Drum: A rhythmic approach to interactive analytics on large data | |
| US9280384B2 (en) | Method, server and system for processing task data | |
| CN115829579A (en) | Resource data processing method, device, server and storage medium | |
| CN113553203A (en) | Request processing method, device, server and storage medium | |
| CN115604041B (en) | Security agent method, system, apparatus, computer device, and storage medium | |
| CN112650778B (en) | Information recommendation system, method, device, server and storage medium | |
| US11887154B2 (en) | Model for serving exploration traffic | |
| CN118075059B (en) | Gateway execution method, device and equipment based on MySQL protocol | |
| CN115297336B (en) | Task processing method and device, electronic equipment, storage medium and program product | |
| CN110442447B (en) | Message queue-based load balancing method and device and computer equipment | |
| CN113961744A (en) | Live broadcast cover information processing method and device, electronic equipment and storage medium | |
| CN114896074A (en) | Resource updating method and device, electronic equipment and storage medium | |
| CN117056381A (en) | Virtual article pushing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |