CN113127732B - Service data acquisition method, device, computer equipment and storage medium - Google Patents
Service data acquisition method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN113127732B CN113127732B CN202110307293.8A CN202110307293A CN113127732B CN 113127732 B CN113127732 B CN 113127732B CN 202110307293 A CN202110307293 A CN 202110307293A CN 113127732 B CN113127732 B CN 113127732B
- Authority
- CN
- China
- Prior art keywords
- data
- client
- search
- server
- long connection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 238000013515 script Methods 0.000 claims abstract description 25
- 230000008859 change Effects 0.000 claims description 67
- 230000002159 abnormal effect Effects 0.000 claims description 23
- 238000012545 processing Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 13
- 238000012790 confirmation Methods 0.000 claims description 5
- 230000000977 initiatory effect Effects 0.000 claims description 4
- 238000007726 management method Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 6
- 238000007792 addition Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000001914 filtration Methods 0.000 description 4
- 230000000903 blocking effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008520 organization Effects 0.000 description 3
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000010354 integration Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9535—Search customisation based on user profiles and personalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/953—Querying, e.g. by the use of web search engines
- G06F16/9538—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/957—Browsing optimisation, e.g. caching or content distillation
- G06F16/9574—Browsing optimisation, e.g. caching or content distillation of access to content, e.g. by caching
Abstract
The application relates to a service data acquisition method, a service data acquisition device, computer equipment and a storage medium. The method comprises the following steps: acquiring a data search request initiated by a user; the data search request comprises search conditions; according to the search conditions, searching basic data in a local buffer of the client by utilizing language scripts of the client to obtain search results conforming to the search conditions; the basic data is the business data downloaded from the business server in advance and stored in the local buffer; and displaying the search result in a browser interface. The method can effectively improve the acquisition efficiency of the service data.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for acquiring service data, a computer device, and a storage medium.
Background
With the development of computer technology and the advent of the 5G age, the appearance of the Internet brings great convenience to modern life, and more enterprises can process various services on line by using a system service platform, so that convenience is brought to users. Particularly in large systems involving group users, such as banks, the hundreds of millions of related data carried by the large systems are very important problems in the industries, such as how to quickly and accurately search for effective data. In the traditional service data acquisition mode, data filtering and access are realized by traversing the data in different dimensions in the query server database.
However, in the current method for acquiring service data, the data of the database is generally read in real time based on the HTTP interface, and then the server returns the result meeting the search condition to the client for display after filtering and sorting the keywords, but frequent access to the database may cause high concurrency related performance bottlenecks, especially in the case of poor network environment or large data volume, which easily results in low service data acquisition efficiency.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a service data acquisition method, apparatus, computer device, and storage medium that can improve the efficiency of service data acquisition.
A method for acquiring service data, the method comprising:
acquiring a data search request initiated by a user; the data search request comprises search conditions;
according to the search conditions, searching basic data in a local buffer of the client by utilizing language scripts of the client to obtain search results conforming to the search conditions; the basic data is the business data downloaded from the business server in advance and stored in the local buffer;
and displaying the search result in a browser interface.
In one embodiment, before the acquiring the data search request initiated by the user, the method further includes:
initiating an asynchronous data acquisition request to a service server; the asynchronous data acquisition request carries an account set identifier;
receiving basic data returned by the service server according to the account set identifier;
and storing the basic data into the local buffer of the client.
In one embodiment, the method further comprises:
receiving a notification message pushed by a long connection server through a long connection channel;
acquiring the message type of the notification message;
and acquiring corresponding change data according to the message type, and updating the change data to the local buffer of the client.
In one embodiment, the method further comprises:
responding to the change operation of the user, and uploading data corresponding to the change operation to a service server;
the receiving the notification message pushed by the long connection server through the long connection channel includes:
and receiving a notification message pushed by the long connection server through the long connection channel, wherein the notification message is sent to the long connection server by the service server, and the notification message is a notification message carrying the data change type and generated according to the data change type by determining the data change type when the service server monitors the data change.
In one embodiment, the method for establishing the long connection channel includes:
sending a long connection request to a long connection server; the long connection request carries the negotiated secret key, encryption mode and channel management mode;
and receiving confirmation information which is returned by the long connection server and used for establishing a long connection channel with the client according to the negotiated secret key, the encryption mode and the channel management mode.
In one embodiment, the client is provided with a heartbeat checking mode;
the method further comprises the steps of:
periodically checking the condition of each node by the heartbeat checking mode;
and when the abnormal situation is checked, reporting the abnormal situation to a server to instruct the server to switch to the traditional mode search data.
In one embodiment, the periodically checking each node through the heartbeat checking mode includes:
comparing the time stamps of the database operation, and accessing the designated node port in a polling mode to obtain a corresponding access result;
and judging whether the node port is abnormal or not according to the access result.
An apparatus for acquiring service data, the apparatus comprising:
The acquisition module is used for acquiring a data search request initiated by a user; the data search request comprises search conditions;
the search processing module is used for carrying out search processing on the basic data in the local buffer of the client by utilizing language scripts according to the search conditions to obtain search results conforming to the search conditions; the basic data is the business data downloaded from the business server in advance and stored in the local buffer;
and the display module is used for displaying the search result in the browser interface.
A computer device comprising a memory storing a computer program and a processor which when executing the computer program performs the steps of:
acquiring a data search request initiated by a user; the data search request comprises search conditions;
according to the search conditions, searching basic data in a local buffer of the client by utilizing language scripts of the client to obtain search results conforming to the search conditions; the basic data is the business data downloaded from the business server in advance and stored in the local buffer;
and displaying the search result in a browser interface.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
acquiring a data search request initiated by a user; the data search request comprises search conditions;
according to the search conditions, searching basic data in a local buffer of the client by utilizing language scripts of the client to obtain search results conforming to the search conditions; the basic data is the business data downloaded from the business server in advance and stored in the local buffer;
and displaying the search result in a browser interface.
According to the method, the device, the computer equipment and the storage medium for acquiring the service data, the data search request initiated by the user is acquired, the data search request contains the search condition, basic data in the local buffer of the client is searched and processed according to the search condition by using the language script of the client to obtain the search result meeting the search condition, the basic data is the service data which is downloaded from the service server in advance and stored in the local buffer, and the search result is displayed in the browser interface. Therefore, frequent interaction with the server is not needed, data query capability of the server is not needed, business cache objects based on the browser are not needed, real-time query is not needed to depend on an unstable network environment, query is efficiently executed in a local buffer by using language scripts of the browser, the efficiency of real-time search of a user can be improved, the result of the actual need search of the user is obtained in the most efficient mode, and the obtaining efficiency of business data is greatly improved.
Drawings
FIG. 1 is an application environment diagram of a method for acquiring service data in one embodiment;
FIG. 2 is a flow chart of a method for acquiring service data in one embodiment;
FIG. 3 is a flowchart illustrating steps for storing base data in a local buffer of a client according to one embodiment;
FIG. 4 is a flowchart illustrating steps for updating change data to a local buffer of a client according to an embodiment;
FIG. 5 is a schematic flow chart of processing service data among a client, a service server and a long connection server in one embodiment;
FIG. 6 is a timing diagram of a service call when a long connection user changes traffic data, in one embodiment;
FIG. 7 is a block diagram of an apparatus for acquiring service data in one embodiment;
fig. 8 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
The service data acquisition method provided by the application can be applied to an application environment shown in fig. 1. Wherein the client 102 communicates with the service server 104 over a network. The client 102 obtains a data search request initiated by a user, wherein the data search request contains search conditions, and the client 102 performs search processing on basic data in a local buffer of the client by using language scripts according to the search conditions to obtain search results conforming to the search conditions; the basic profile data is service data downloaded from the service server 104 in advance and stored in the local buffer, and the client 102 displays the search result in the browser interface. The client 102 may be, but not limited to, various personal computers, notebook computers, smartphones, tablet computers, and portable wearable devices, and the service server 104 may be implemented by a stand-alone server or a server cluster composed of a plurality of servers.
In one embodiment, as shown in fig. 2, a method for acquiring service data is provided, and the method is applied to the client in fig. 1 for illustration, and includes the following steps:
step 202, obtaining a data search request initiated by a user, wherein the data search request comprises search conditions.
By adopting a unified service management information platform, each enterprise can uniformly manage all resources and information on the enterprise interior and the enterprise exterior supply chain, the integration can eliminate various information barriers and information islands caused by division in the enterprise interior, for example, a developer can adopt a micro-service architecture to create applications of different service functions around service field components, the applications can be independently developed, managed and updated, and the deployment, management and service function delivery can be simpler by using the micro-service cloud architecture and the platform in the scattered components. BBOC (browser-based Business Object Cache) refers to browser-based business object caching, and has low requirements on computer configuration and network requirements of users, and conventional computers can be supported. The method and the device can be used for quickly and effectively acquiring the service data needed by the user based on the service object cache of the browser.
Specifically, the client may obtain a data search request initiated by the user, where the data search request includes a search condition. Namely, when the user initiates a data search request in the browser through triggering operation, the client can acquire the data search request initiated by the user, wherein the data search request contains search conditions. The client or the user terminal refers to a program corresponding to the server and providing local service for the user. Except for some applications that only run locally, they are typically installed on a common client and need to run in conjunction with the server. Clients may include DNS clients (DNS is an abbreviation for domain name system, domain Name System), web clients, mobile clients, and the like. For example, the mobile client uses a mobile phone as an example, and the mobile client is software that can run on the mobile phone terminal. The data search request refers to a service data search request corresponding to a service bill, which is initiated by a user in different service scene systems according to different requirements. For example, when performing a recording operation, the user needs to query merchandise information corresponding to different business documents. The search condition refers to a combination of one or more tag information input by a user for screening out target information.
And 204, according to the search conditions, searching basic data in the local buffer of the client by using language scripts to obtain search results meeting the search conditions, wherein the basic data is service data downloaded from the service server in advance and stored in the local buffer.
After the client obtains the data search request initiated by the user, the client can search the basic data in the local buffer of the client by using the language script according to the search condition contained in the data search request, so as to obtain the search result meeting the search condition. The basic data is the service data downloaded from the service server in advance and stored in the local buffer. Language script refers to a programming language script that the client is self-contained, such as js language script. The underlying data refers to business data associated with the business document.
Specifically, a user can log in a service system of a specific scene by inputting a user name and a password in a browser webpage, the user can initiate a specific data acquisition request through an app (Application) client or a web client, namely a web browser, and a service system, namely a service server, can receive the data acquisition request sent by the user through the client and pull corresponding basic data from a database according to identification information in the data acquisition request to return to the client. After receiving the basic data returned by the service server, the client can store the basic data in a local buffer. Further, when the user initiates the data search request through triggering operation, the client can acquire the data search request initiated by the user, and search basic data in the local buffer of the client by using language scripts according to the search condition contained in the data search request, so as to obtain a search result conforming to the search condition. For example, the client performs search processing on the basic data in the local buffer of the client by using js language script according to the search condition included in the data search request, so as to obtain a search result conforming to the search condition.
Step 206, displaying the search results in the browser interface.
The client performs search processing on the basic data in the local buffer of the client by using language scripts according to the search conditions in the data search request, and after obtaining the search results meeting the search conditions, the client can directly display the search results in a browser interface so as to facilitate the quick check of users. The user may preset a user-defined display manner, for example, tiling or list display, etc., and it is understood that the manner of displaying the search result in the browser interface in the present application is not limited to tiling or list display, but may also include other manners.
In the traditional service data acquisition mode, data of a database is generally read in real time based on an HTTP interface, and then a server returns a result meeting search conditions to a client for display after filtering and sequencing keywords, so that high concurrency related performance bottlenecks can be brought by frequently establishing an HTTP request channel, network safety and stability of a system are not facilitated, and the service data acquisition efficiency is easy to be low especially in an environment with poor network environment or poor client configuration used by a user.
In this embodiment, the data search request initiated by the user is obtained, where the data search request includes a search condition, and according to the search condition, basic data in the local buffer of the client is searched by using a language script of the user to obtain a search result that meets the search condition, where the basic data is service data downloaded from the service server in advance and stored in the local buffer, and the search result is displayed in the browser interface. Therefore, frequent interaction with the server is not needed, data query capability of the server is not depended, real-time query is not depended on unstable network environment based on the business cache object of the browser, the query can be efficiently executed in the local buffer by utilizing the language script of the browser, the efficiency of real-time search of the user can be improved, the result of the search actually needed by the user is obtained in the most efficient mode, and the obtaining efficiency of business data is greatly improved.
In one embodiment, as shown in fig. 3, before obtaining the data search request initiated by the user, the method further includes a step of storing the basic material data in the local buffer of the client, and specifically includes:
Step 302, an asynchronous data acquisition request is initiated to a service server, wherein the asynchronous data acquisition request carries an account cover identifier.
And step 304, receiving the basic data returned by the service server according to the account cover identification.
Step 306, store the basic data into the local buffer of the client.
Before the client obtains the data search request initiated by the user, the client can store the pulled basic data into the local buffer of the client. Specifically, after the user logs in the client, the client may initiate an asynchronous data acquisition request to the service server, where the asynchronous data acquisition request carries the account cover identifier. And the client receives the basic data returned by the service server according to the account cover identification, and stores the basic data into a local buffer of the client.
Wherein, the asynchronous data acquisition request refers to that the data acquisition request can adopt an asynchronous mode. The account cover identification is used for identifying unique account cover information, different account cover information can be contained in tenant information, and different user information can be contained in account cover information. Corresponding tenant information can be created according to different enterprise types in the business system, business data among tenants are completely isolated, and the business data can comprise business service data, work order data, organization data and the like. For the same tenant, the service system partitions the user management center and the application management center corresponding to the tenant, and the user management center uses a unified set of organization, role information and user information, namely, the data of the organization, the role, the user and the like which are uniformly set, and can be shared by all departments in the same tenant. Different departments of the tenant, namely different account groups, can establish own business services of the departments at an application management center interface by logging in the management account number. Each tenant can create business application services belonging to different roles under its own organizational framework.
For example, after a user logs in the client, the client may initiate an asynchronous non-blocking (silence) request to the server to pull the corresponding basic data, and the client may compress and encrypt the pulled basic data according to a preset symmetric or asymmetric encryption manner, and store the compressed and encrypted basic data in a local buffer of the client for subsequent query. In the traditional way of reading database data in real time based on the HTTP interface, the browser has 6 channels based on the HTTP request, and frequently establishes the HTTP request channel easily causes that the front request can block the operation of other requests, but in the embodiment, based on the business cache object of the browser, the real-time query is not dependent on unstable network environment, the real-time query is efficiently executed in js script of the browser, and even under the condition of hundreds of thousands of equivalent data, the response speed can reach hundreds of milliseconds. By initiating asynchronous non-blocking (silent) requests to pull corresponding basic data, normal loading of other requests of the webpage is not affected, support is provided for the follow-up acceleration of first screen display, network condition support is only needed when synchronous data or data are pulled, frequent interaction with a server is not needed, data query capacity of the server is not relied on, and the efficiency of real-time searching of a user can be effectively improved.
In one embodiment, the method further comprises the step of uploading data corresponding to the change operation to the service server in response to the change operation of the user. Specifically, when the user subsequently performs a change operation in the service system, (possibly a plurality of clients, for example, a scene of multi-account set), that is, when the user triggers the change of the basic data through a trigger operation, the client responds to the change operation of the user and uploads the data corresponding to the change operation to the service server. The multi-account-cover scene means that one business system can have a plurality of account covers at the same time, and a client of each account cover can change the data in the business system. The types of change operations may include new additions, modifications, deletions, batch additions, batch modifications, and batch deletions. For example, the user may perform a batch addition operation on certain merchandise information, and the client responds to the batch addition operation of the user and uploads data corresponding to the batch addition operation to the service server. When the service server monitors the change of the corresponding basic data (commodity information), the service server judges that the change type is a batch new operation according to the change condition, generates a corresponding notification message, and sends the notification message (including the message type is incremental update) to the long connection server, so that the long connection server pushes the notification message to the corresponding client through the long connection channel to instruct the client to update the data in the local buffer of the client correspondingly. Therefore, the state of the basic data is monitored in real time through the service server, when the change of the basic data is monitored, the corresponding notification message can be timely sent to the long connection server and pushed to the corresponding client through the long connection server, and therefore the accuracy, instantaneity and safety of the data in the local buffer of the client are guaranteed.
In one embodiment, as shown in fig. 4, the method further includes a step of updating the change data into the local buffer of the client, specifically including:
step 402, receiving a notification message pushed by the long connection server through the long connection channel.
Step 404, obtaining the message type of the notification message.
Step 406, according to the message type, obtaining the corresponding change data, and updating the change data to the local buffer of the client.
When the service server monitors the change of the basic data, the service server generates a corresponding notification message according to the change condition and sends the notification message (including the message type) to the long connection server. The message queue of the long connection server consumes the messages sent by each service server node in real time and pushes the messages to each corresponding long connection client, so that the client obtains corresponding change information in an increment mode according to the message type of the received messages, and updates the data stored in the buffer according to the change information. The Message Queue (MQ) is a container for storing messages during the transmission process of the messages. The message queue can be deployed on the cloud platform and mainly used for receiving and sending messages, and when facing a large amount of task information, the message queue can cut peaks and smooth valleys of the messages.
Specifically, the client receives a notification message pushed by the long connection server through the long connection channel. Further, the client acquires the message type of the received notification message. And the client acquires corresponding change data according to the message type and updates the change data to the local buffer of the client. The message types may include delta updates and full changes, among others. Therefore, the data multi-terminal synchronization can be realized by the mode of acquiring the change information in an increment, more time cost is saved, the service load is reduced, the downtime is reduced, and stable service is ensured.
In one embodiment, the step of receiving a notification message pushed by the long connection server through the long connection channel includes:
and receiving a notification message pushed by the long connection server through the long connection channel, wherein the notification message is sent to the long connection server by the service server, and when the service server monitors the data change, the notification message determines the type of the data change and generates a notification message carrying the data change type according to the type of the data change.
The message queues of the long connection servers consume messages sent by each service server node in real time and push the messages to each corresponding long connection client. Specifically, the client receives a notification message pushed by the long connection server through the long connection channel, wherein the notification message is sent to the long connection server by the service server, the notification message is a notification message which is generated to carry the data change type according to the data change type and the type of the data change is determined when the service server monitors the data change. The service server monitors operation actions in real time, namely the service server monitors operation affecting results in real time, when the service server monitors service data change, the service server judges the basic type of the service data change by establishing an analysis model, sorts data packets, and finally sends the data packets to the long connection server to generate corresponding consumption type information. Therefore, a stable long connection channel is established between the client and the long connection server, and meanwhile, the service server monitors the change of basic data in real time, so that the function of updating information of the client in real time is enhanced, and the accuracy, the instantaneity and the safety of data in a local buffer of the client are ensured.
In one embodiment, the step of establishing a long connection channel includes:
and sending a long connection request to the long connection server, wherein the long connection request carries the negotiated secret key, encryption mode and channel management mode.
And receiving confirmation information which is returned by the long connection server and used for establishing a long connection channel with the client according to the negotiated secret key, the encryption mode and the channel management mode.
After a user logs in a client, the client initiates an asynchronous non-blocking (silent) request to a server to pull corresponding basic data, and after the pulled basic data is stored in the local of the client, the client can send a long connection request to a long connection server, wherein the long connection request carries a negotiated secret key, an encryption mode and a channel management mode. The client receives confirmation information of establishing a long connection channel with the client returned by the long connection server according to the negotiated secret key, encryption mode and channel management mode. The long connection service can be independently deployed on other servers, and the long connection server in the application is a high-reliability long connection message service system and supports the characteristics of message queue, high concurrency and the like based on the mode design of a producer-consumer. And the internal parts of the messages of the servers are combined and are pushed to a message queue for processing by adopting a production-consumption mode. The client receives a large amount of messages in unit time and performs merging post-processing, and each client has only one socket case, so that the throughput of the client for actually processing the messages is improved. In addition, when the network fluctuates, the long connection channel may be disconnected, and the client may automatically perform the long connection reconnection operation according to a preset rule. When the long connection is disconnected, the client continuously tries reconnection according to an increasing time interval, the reconnection exceeds a preset threshold number (for example, 10 times), if the reconnection still cannot be connected, the switch is switched to an old mode, and the data is pulled in full quantity, so that the accuracy of the data is ensured.
In this embodiment, the long connection server and the client establish a unique long connection channel, which relates to a design of a room layer concept, so as to ensure uniqueness of the client, that is, the long connection server generates a json file according to key, encryption mode, channel and other elements, and performs unified management in the long connection server. In this embodiment, a long connection (websocket) channel request related to a long connection service is initiated, and a unique channel (uid) is established with each client according to negotiated information such as a key, an encryption mode, channel management, and the like, so that a subsequent client can receive message pushing in real time, thereby ensuring accuracy, instantaneity and security of data in a local buffer of the client.
In one embodiment, the client is provided with a heartbeat checking mode, and the method further includes a step of reporting the abnormal condition to the server, specifically including:
each node is checked periodically by means of heartbeat check.
When the abnormal situation is checked, the abnormal situation is reported to the server to instruct the server to switch to the traditional mode search data.
The long-connection push service based on the BBOC has long links of the whole system and needs to coordinate a plurality of nodes, so that a heartbeat detection mechanism is arranged, when the links have problems, the system can be switched to an old mode to ensure the service availability, and meanwhile, the abnormal condition of the nodes can be timely reported to a business server headquarter to check the problem root. Specifically, the client can periodically check the condition of each node through a heartbeat check mode, and when abnormal conditions occur in the check, the abnormal conditions are reported to the server to instruct the server to switch to the traditional mode search data. For example, when the heartbeat check interface hangs up, the check is ignored, and when a preset threshold number of check failures (e.g. 10) are exceeded, the client statistically reports the exception and tries to switch the old mode, in which state the service can still continue to operate. Therefore, when abnormal conditions occur, the service can be reported in time, the service is ensured to be uninterrupted, the mode of switching the old mode is adopted, and the high availability of the whole system is ensured.
In one embodiment, the step of periodically checking the condition of each node by means of heartbeat checking includes:
and comparing the timestamps of the database operation, and accessing the designated node ports in a polling mode to obtain a corresponding access result.
And judging whether the node port is abnormal or not according to the access result.
The client can check the condition of each node periodically by a heartbeat check mode. Specifically, the client can access the designated node port in a polling mode by comparing the time stamps of the database operation to obtain a corresponding access result. Further, the client can determine whether the node port is abnormal according to the access result. For example, the client can access the port designated by the server through an api polling mode to obtain a corresponding access result. The client can judge whether the node port is abnormal or not according to the access result. Therefore, when abnormal conditions occur, the system can report in time, ensure uninterrupted service and ensure high availability of the whole system.
In one embodiment, as shown in fig. 5, a flow chart of processing service data between a client, a service server and a long connection server is shown. Wherein, the business server is provided with basic data service and monitor. The client is locally deployed with a buffer. The message push Long service is deployed in a Long connection server. Specifically, after the user logs in the client, the client pulls the corresponding basic data through the Api service and writes the basic data into the cache. Further, the client sends a request to the message push Long service, and a Long connection channel corresponding to the message push Long service is established. When the user searches commodity data in the list, the client can acquire a data search request initiated by the user, and search basic data in a local buffer of the client by utilizing language scripts according to search conditions contained in the data search request to obtain search results meeting the search conditions, namely, the client displays the search data corresponding to the search results in the buffer in a browser interface for the user to quickly check, and the list recording operation is completed. Based on the data in each client buffer, the script language of the client is used for efficiently filtering, matching and sequencing out the search content meeting the requirements, so that the search is more efficient.
Further, when the listener of the service server listens to the change of the basic data, the message with different message types is produced and sent to the message pushing Long service, and the message pushing Long service can consume the messages with different types and push the messages to all the same-frequency clients. The same-frequency client refers to a client that receives push messages based on the same long connection channel, for example, clients corresponding to different users under the same account cover. And the client acquires the change of the cache data in a pull alignment mode according to the received push messages of different types. Pull alignment refers to that there may be some difference in the data cached in each browser, for example, if the a client updates the data, the B client needs to update the data in real time to keep consistent, and this consistent process is pull alignment. The client tries to pull the alignment for a plurality of times, when the api interface is sporadically abnormal, the client tries to acquire the service data for a preset number of times (for example, three times) until the request is successful, the accuracy of pulling the basic data alignment is ensured, and if the api interface is completely hung up, the client records and reports the triggering time and the request information. Therefore, a high-concurrency, high-availability and monitorable service data acquisition scheme is realized, so that the user searching smoothness is higher, the user can input and check instantly, and the ordering is more efficient.
In addition, the client can update the data in the local cache of the browser in an increment mode and a full-quantity mode according to a preset model, for example, if a single commodity is newly added, the client can update the data in the local cache in an increment mode, and if a batch of commodities are newly added, the client can update the data in the local cache in a full-quantity mode. After updating the data in the local cache of the browser, the client can periodically check the alignment of the data and the condition of the closed link by a heartbeat check mode, and when the abnormal condition occurs in the check, the abnormal condition is reported to the server so as to instruct the server to switch to the traditional mode for processing the data. Therefore, based on the data in each client buffer, the script language of the client can efficiently filter, match and sort out the search content meeting the requirements, the network is not required to be requested, the time cost required for inquiring the server background database is reduced to the client browser, the more efficient search efficiency is realized, and the service data acquisition efficiency is greatly improved.
In one embodiment, as shown in fig. 6, a timing diagram is invoked for a service when a long connection user changes traffic data. The BD service is basic data service management, is used for uniformly managing change data, and can be deployed in a service server; rabbitMQ is an open source message proxy and message queue server for handling message deployment in long connection servers; tf-appise refers to a service message notification, which is used to uniformly issue a request to a long connection service, and can be deployed in a service server; long service is a Long connection service, and a service for guaranteeing real-time data can be deployed in a Long connection server. The long connection service (WebSocket) supports a high-concurrency long connection push service, and has a complete message queue management mechanism and excellent data throughput capacity. The Web buffer SDK supports integration of the direct-out technology across a plurality of terminals, and has the characteristics of high availability, good expansibility and good compatibility.
Specifically, the user may perform a change operation on the basic data, that is, when the BD service monitors that the basic data is changed, the change message is synchronously sent to the message queue server, the message queue server may asynchronously consume the message, push the notification message to the long connection service through the service message notification, so that the long connection service pushes the notification message to the corresponding client through the long connection channel, and the client may request to obtain the corresponding basic data from the BD service, that is, the service server, according to the received push message, and the service server returns the corresponding incremental basic data according to the request. In addition, the developer can also link up specific operation logic of different services on the basis of processing the message by the long connection service queue, so that when the message is pushed to the client, the step of querying an interface by the client through the api is not needed, namely, the long connection service needs to integrate more service operation logic. The association logic of the decoupling service and the push service is easier to maintain, each node which can be monitored ensures the stability of the whole system, can realize multi-terminal synchronization of data, and saves more time cost.
It should be understood that, although the steps in the flowcharts of fig. 1-6 are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in FIGS. 1-6 may include multiple steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor do the order in which the steps or stages are performed necessarily performed in sequence, but may be performed alternately or alternately with at least a portion of the steps or stages in other steps or other steps.
In one embodiment, as shown in fig. 7, there is provided a service data acquisition apparatus, including: an acquisition module 702, a search processing module 704, and a display module 706, wherein:
the obtaining module 702 is configured to obtain a data search request initiated by a user, where the data search request includes a search condition.
And the search processing module 704 is configured to perform search processing on basic data in the local buffer of the client by using language scripts according to the search conditions, so as to obtain a search result that meets the search conditions, where the basic data is service data downloaded from the service server in advance and stored in the local buffer.
And the display module 706 is configured to display the search result in the browser interface.
In one embodiment, the apparatus further comprises: the device comprises a sending module, a receiving module and a storage module.
The sending module is used for initiating an asynchronous data acquisition request to the service server, wherein the asynchronous data acquisition request carries an account cover identifier. The receiving module is used for receiving the basic data returned by the service server according to the account set identification. The storage module is used for storing the basic data into the local buffer of the client.
In one embodiment, the apparatus further comprises: and updating the module.
The receiving module is also used for receiving the notification message pushed by the long connection server through the long connection channel. The acquisition module is also used for acquiring the message type of the notification message. The updating module is used for acquiring corresponding change data according to the message type and updating the change data to the local buffer of the client.
In one embodiment, the apparatus further comprises: and uploading the module.
The uploading module is used for responding to the changing operation of the user and uploading the data corresponding to the changing operation to the service server. The receiving module is further configured to receive a notification message pushed by the long connection server through the long connection channel, where the notification message is sent by the service server to the long connection server, and the notification message is a notification message that the service server determines a type of data change when monitoring the data change, and generates a notification message carrying the type of data change according to the type of data change.
In one embodiment, the sending module is further configured to send a long connection request to the long connection server, where the long connection request carries the negotiated key, encryption scheme, and channel management scheme. The receiving module is also used for receiving confirmation information which is returned by the long connection server and used for establishing a long connection channel with the client according to the negotiated secret key, the encryption mode and the channel management mode.
In one embodiment, the apparatus further comprises: and (5) checking the module.
The checking module is used for checking the condition of each node periodically in a heartbeat checking mode; when the abnormal situation is checked, the abnormal situation is reported to the server to instruct the server to switch to the traditional mode search data.
In one embodiment, the apparatus further comprises: an access module and a decision module.
The access module is used for comparing the timestamps of database operations, and accessing the designated node ports in a polling mode to obtain corresponding access results. The judging module is used for judging whether the node port is abnormal or not according to the access result.
For specific limitation of the service data acquisition device, reference may be made to the limitation of the service data acquisition method hereinabove, and the description thereof will not be repeated here. The above-mentioned various modules in the service data acquisition device may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure thereof may be as shown in fig. 8. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program, when executed by a processor, implements a method of acquiring service data. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in fig. 8 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided that includes a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the various method embodiments described above when the computer program is executed.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, or the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (10)
1. A method for acquiring service data, applied to a client, the method comprising:
acquiring a data search request initiated by a user; the data search request comprises search conditions;
according to the search conditions, searching basic data in a local buffer of the client by utilizing language scripts of the client to obtain search results conforming to the search conditions; the basic data is the business data downloaded from the business server in advance and stored in the local buffer;
Displaying the search result in a browser interface;
when another client under the same account cover with the client responds to the change operation on the basic data, uploading the data corresponding to the change operation to the service server, and receiving a notification message pushed by the long connection server through a long connection channel; the notification message is generated and sent to the long connection server when the service server monitors the change of the basic data;
and acquiring corresponding change data from the service server in a pull alignment mode according to the message type of the notification message, and updating the change data to the local buffer of the client so that the data buffered by all the clients under the same account cover are kept consistent in real time.
2. The method of claim 1, wherein prior to the obtaining the user initiated data search request, the method further comprises:
initiating an asynchronous data acquisition request to the service server; the asynchronous data acquisition request carries an account set identifier;
receiving basic data returned by the service server according to the account set identifier;
And storing the basic data into the local buffer of the client.
3. The method of claim 1, wherein the notification message contains a message type, the message type including delta updates and full changes.
4. A method according to claim 3, wherein the notification message is a notification message generated by the service server carrying a type of data change by determining the type of data change when the service server monitors the data change and based on the type of data change.
5. A method according to claim 3, wherein the long connection channel is established by a method comprising:
sending a long connection request to a long connection server; the long connection request carries the negotiated secret key, encryption mode and channel management mode;
and receiving confirmation information which is returned by the long connection server and used for establishing a long connection channel with the client according to the negotiated secret key, the encryption mode and the channel management mode.
6. The method according to claim 1, wherein the client is provided with a heartbeat check mode;
the method further comprises the steps of:
periodically checking the condition of each node by the heartbeat checking mode;
And when the abnormal situation is checked, reporting the abnormal situation to a server to instruct the server to switch to the traditional mode search data.
7. The method of claim 6, wherein periodically checking the condition of each node by the heartbeat check mode comprises:
comparing the time stamps of the database operation, and accessing the designated node port in a polling mode to obtain a corresponding access result;
and judging whether the node port is abnormal or not according to the access result.
8. A device for acquiring service data, applied to a client, the device comprising:
the acquisition module is used for acquiring a data search request initiated by a user; the data search request comprises search conditions;
the search processing module is used for carrying out search processing on the basic data in the local buffer of the client by utilizing language scripts according to the search conditions to obtain search results conforming to the search conditions; the basic data is the business data downloaded from the business server in advance and stored in the local buffer;
the display module is used for displaying the search results in a browser interface;
The updating module is used for receiving a notification message pushed by the long connection server through the long connection channel when another client side under the same account cover with the client side responds to the change operation on the basic data and uploads the data corresponding to the change operation to the service server; the notification message is generated and sent to the long connection server when the service server monitors the change of the basic data; and acquiring corresponding change data from the service server in a pull alignment mode according to the message type of the notification message, and updating the change data to the local buffer of the client so that the data buffered by all the clients under the same account cover are kept consistent in real time.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110307293.8A CN113127732B (en) | 2021-03-23 | 2021-03-23 | Service data acquisition method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110307293.8A CN113127732B (en) | 2021-03-23 | 2021-03-23 | Service data acquisition method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113127732A CN113127732A (en) | 2021-07-16 |
| CN113127732B true CN113127732B (en) | 2024-02-20 |
Family
ID=76773750
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110307293.8A Active CN113127732B (en) | 2021-03-23 | 2021-03-23 | Service data acquisition method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113127732B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113645294B (en) * | 2021-08-06 | 2022-08-26 | 腾讯科技（深圳）有限公司 | Message acquisition method and device, computer equipment and message transmission system |
| CN113747127A (en) * | 2021-11-08 | 2021-12-03 | 西安热工研究院有限公司 | DCS monitoring image measuring point positioning method, system, equipment and storage medium |
| CN114020321B (en) * | 2021-11-09 | 2023-08-18 | 江苏点石乐投科技有限公司 | Intelligent application interface making system and method based on user behavior data |
| CN115051956B (en) * | 2022-06-30 | 2023-09-26 | 北京达佳互联信息技术有限公司 | Connection establishment method, device, equipment and storage medium |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104615651A (en) * | 2014-12-26 | 2015-05-13 | 北京奇虎科技有限公司 | Service information processing method and browser client |
| CN105657444A (en) * | 2014-11-25 | 2016-06-08 | 中兴通讯股份有限公司 | Query and search method, device and system for business data |
| CN106156061A (en) * | 2015-03-30 | 2016-11-23 | 北大方正集团有限公司 | A kind of method and device improving efficiency data query |
| CN108470043A (en) * | 2018-02-27 | 2018-08-31 | 阿里巴巴集团控股有限公司 | A kind of acquisition methods and device of business result |
| CN108733685A (en) * | 2017-04-17 | 2018-11-02 | 阿里巴巴集团控股有限公司 | Object interaction, browsing, search and configuration method and device |
| CN109660584A (en) * | 2017-10-12 | 2019-04-19 | 阿里巴巴集团控股有限公司 | A kind of method and communication means and communication system of client and the long connection of server foundation |
| US10534783B1 (en) * | 2016-02-08 | 2020-01-14 | Microstrategy Incorporated | Enterprise search |
| CN111177539A (en) * | 2019-12-16 | 2020-05-19 | 北京百度网讯科技有限公司 | Search result page generation method and device, electronic equipment and storage medium |
| CN111400578A (en) * | 2020-03-02 | 2020-07-10 | 金蝶蝶金云计算有限公司 | Goods data query method and device, computer equipment and storage medium |
| CN111581239A (en) * | 2020-04-10 | 2020-08-25 | 支付宝实验室(新加坡)有限公司 | Cache refreshing method and electronic equipment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7752326B2 (en) * | 2001-08-20 | 2010-07-06 | Masterobjects, Inc. | System and method for utilizing asynchronous client server communication objects |
| CN103123593B (en) * | 2011-11-18 | 2016-08-31 | 腾讯科技（深圳）有限公司 | Java application server and the method processing Web service |
-
2021
- 2021-03-23 CN CN202110307293.8A patent/CN113127732B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105657444A (en) * | 2014-11-25 | 2016-06-08 | 中兴通讯股份有限公司 | Query and search method, device and system for business data |
| CN104615651A (en) * | 2014-12-26 | 2015-05-13 | 北京奇虎科技有限公司 | Service information processing method and browser client |
| CN106156061A (en) * | 2015-03-30 | 2016-11-23 | 北大方正集团有限公司 | A kind of method and device improving efficiency data query |
| US10534783B1 (en) * | 2016-02-08 | 2020-01-14 | Microstrategy Incorporated | Enterprise search |
| CN108733685A (en) * | 2017-04-17 | 2018-11-02 | 阿里巴巴集团控股有限公司 | Object interaction, browsing, search and configuration method and device |
| CN109660584A (en) * | 2017-10-12 | 2019-04-19 | 阿里巴巴集团控股有限公司 | A kind of method and communication means and communication system of client and the long connection of server foundation |
| CN108470043A (en) * | 2018-02-27 | 2018-08-31 | 阿里巴巴集团控股有限公司 | A kind of acquisition methods and device of business result |
| CN111177539A (en) * | 2019-12-16 | 2020-05-19 | 北京百度网讯科技有限公司 | Search result page generation method and device, electronic equipment and storage medium |
| CN111400578A (en) * | 2020-03-02 | 2020-07-10 | 金蝶蝶金云计算有限公司 | Goods data query method and device, computer equipment and storage medium |
| CN111581239A (en) * | 2020-04-10 | 2020-08-25 | 支付宝实验室(新加坡)有限公司 | Cache refreshing method and electronic equipment |
Non-Patent Citations (2)
| Title |
|---|
| 蒋文龙.金财互联固定资产子系统的设计与实现. 中国优秀硕士学位论文全文数据库信息科技辑.2019,第25-69页. * |
| 金财互联固定资产子系统的设计与实现;蒋文龙;中国优秀硕士学位论文全文数据库信息科技辑;第25-69页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113127732A (en) | 2021-07-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113127732B (en) | Service data acquisition method, device, computer equipment and storage medium | |
| US10560465B2 (en) | Real time anomaly detection for data streams | |
| US11392416B2 (en) | Automated reconfiguration of real time data stream processing | |
| US10944628B2 (en) | Network operation support system and network device management method | |
| US9514387B2 (en) | System and method of monitoring and measuring cluster performance hosted by an IAAS provider by means of outlier detection | |
| US9197514B2 (en) | Service level agreement based storage access | |
| US20190294781A1 (en) | Load distribution enabling detection of first appearance of a new property value in pipeline data processing | |
| CN110971655B (en) | Offline client playback and synchronization | |
| US10944655B2 (en) | Data verification based upgrades in time series system | |
| CN111190888A (en) | Method and device for managing graph database cluster | |
| CN113424157A (en) | Multi-dimensional periodic detection of IoT device behavior | |
| CN110837423A (en) | Method and device for automatically acquiring data of guided transport vehicle | |
| US10007562B2 (en) | Business transaction context for call graph | |
| CN108566317A (en) | Business monitoring method, Cloud Server, storage medium and device | |
| CN110661851A (en) | Data exchange method and device | |
| US10659320B2 (en) | Device management system | |
| KR102083666B1 (en) | Server for monitoring server based on cloud computing and method therefor | |
| CN114860782B (en) | Data query method, device, equipment and medium | |
| US20190163392A1 (en) | Lifecycle management of memory devices | |
| CN111294231B (en) | Resource management method and system | |
| CN113778780A (en) | Application stability determination method and device, electronic equipment and storage medium | |
| US10459895B2 (en) | Database storage monitoring equipment | |
| CN113762910A (en) | Document monitoring method and device | |
| CN112433891A (en) | Data processing method and device and server | |
| US11824750B2 (en) | Managing information technology infrastructure based on user experience |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |