CN113515369A - Data processing method, system, terminal and storage medium - Google Patents
Data processing method, system, terminal and storage medium Download PDFInfo
- Publication number
- CN113515369A CN113515369A CN202110440483.7A CN202110440483A CN113515369A CN 113515369 A CN113515369 A CN 113515369A CN 202110440483 A CN202110440483 A CN 202110440483A CN 113515369 A CN113515369 A CN 113515369A
- Authority
- CN
- China
- Prior art keywords
- service
- service request
- server
- data processing
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
- H04L67/63—Routing a service request depending on the request content or context
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Computer And Data Communications (AREA)
- Hardware Redundancy (AREA)
Abstract
The application is applicable to the field of computers and provides a data processing method, a data processing system, a terminal and a storage medium. The data processing method comprises the following steps: acquiring a target service request; the service identification number carried by the target service request is the same as the service identification number corresponding to the service group where the first service server is located; analyzing the target service request to obtain a service message number carried in the target service request; determining a target service processing module corresponding to the service message number in the first service server; and the scheduling target service processing module executes a first data processing operation corresponding to the target service request to obtain a data processing result of the target service request. The method provided by the embodiment of the application can improve the expansibility of the server architecture.
Description
Technical Field
The present application relates to the field of computers, and in particular, to a data processing method, system, terminal, and storage medium.
Background
Data processing is a basic link of system engineering and automatic control, and runs through various fields of social production and social life. Currently, an enterprise providing data information generally performs data processing on a service request of a client used by a user based on a server. However, the traditional server architecture has poor expansibility, which causes a developer to rebuild the server architecture when developing for business changes, and also brings difficulty to the operation and maintenance work of the operation and maintenance personnel.
Disclosure of Invention
The embodiment of the application provides a data processing method, a device, a system, a terminal and a storage medium, which can improve the expansibility of a server architecture.
A first aspect of an embodiment of the present application provides a data processing method, which is applied to a first service server, and the data processing method includes:
acquiring a target service request; the service identification number carried by the target service request is the same as the service identification number corresponding to the service group where the first service server is located;
analyzing the target service request to obtain a service message number carried in the target service request;
determining a target service processing module corresponding to the service message number in the first service server;
and scheduling the target service processing module to execute a first data processing operation corresponding to the target service request to obtain a data processing result of the target service request.
A second aspect of the embodiments of the present application provides a data processing method, which is applied to a scheduling server, where the data processing method includes:
receiving resource occupation data of a third service server sent by the third service server;
if the resource occupation data of the third service server meets a preset load condition, sending a control instruction to the third service server, wherein the control instruction is used for controlling the third service server to stop executing the unfinished service request of the third service server;
and sending the service request which is not completed by the third service server to a first service server which is located in the same service group with the third service server, wherein the data processing method provided by the first service server according to the first aspect of the embodiment of the application performs data processing on the service request which is not completed by the third service server.
A data processing system provided in a third aspect of the embodiments of the present application includes a gateway, a switch, and a first service server;
the gateway is used for receiving a target service request sent by a client, analyzing a service identification number carried in the target service request and sending the target service request to a switch of a message queue with the associated service identification number being the same as the service identification number carried in the target service request;
the switch is used for acquiring the target service request sent by the gateway; acquiring a related target service identification number and positioning information of a service group which is the same as the service identification number carried in the target service request; sending the target service request to a first service server in the service group according to the positioning information;
the first service server is used for acquiring the target service request sent by the switch; analyzing the target service request to obtain a service message number carried in the target service request; determining a target service processing module corresponding to the service message number in the first service server; and scheduling the target service processing module to execute a first data processing operation corresponding to the target service request to obtain a data processing result of the target service request.
A fourth aspect of the present embodiment provides a data processing apparatus, configured in a first service server, where the data processing apparatus includes:
a service request acquiring unit, configured to acquire a target service request; the service identification number carried by the target service request is the same as the service identification number corresponding to the service group where the first service server is located;
a service request analyzing unit, configured to analyze the target service request to obtain a service message number carried in the target service request;
a module determining unit, configured to determine a target service processing module corresponding to the service message number in the first service server;
and the data processing unit is used for scheduling the target service processing module to execute a first data processing operation corresponding to the target service request to obtain a data processing result of the target service request.
A data processing apparatus provided in a fifth aspect of an embodiment of the present application is configured in a scheduling server, and the data processing apparatus includes:
a resource data receiving unit, configured to receive resource occupation data of a third service server sent by the third service server;
a control instruction sending unit, configured to send a control instruction to the third service server if the resource occupation data of the third service server meets a preset load condition, where the control instruction is used to control the third service server to stop executing a service request that is not completed by the third service server;
a service request sending unit, configured to send a service request that is not completed by the third service server to a first service server in the same service group as the third service server, where the first service server performs data processing on the service request that is not completed by the third service server according to the data processing method provided by the first aspect of the embodiment of the present application.
A sixth aspect of the embodiments of the present application provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method when executing the computer program.
A seventh aspect of embodiments of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of the above method.
An eighth aspect of embodiments of the present application provides a computer program product, which when running on a terminal, causes the terminal to implement the steps of the method when executed.
In the embodiment of the application, the servers are divided into different service groups, the first service server can only obtain the target service request with the same target service identification number as the service group where the first service server is located, and perform data processing on the target service request, so that the service server is coupled with the frame and the client in a loose manner, and the two service servers can be independently changed and issued under the condition that interfaces are not changed, namely, a developer can continuously add service servers for responding to the service requests with the same target service identification number in the service groups according to actual needs. Meanwhile, the first service server obtains the service message number corresponding to the target service request by analyzing the target service request, and schedules the target service processing module associated with the service message number to execute the data processing operation corresponding to the target service request, so as to obtain the data processing result of the target service request, namely, each service processing module can respectively process the service requests with the same service message number as the self-associated service message number, so that different service functions executed by each service processing module are not mixed together, and the service processing modules can be carried out concurrently without influencing each other, therefore, developers can add the service processing module in the first service server according to actual conditions, thereby solving the problem of poor expansibility of a service end framework, facilitating development and operation of a service end, reducing the cost of a service system, and saving the investment of operation and maintenance personnel, and the concurrency of the server is also improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic flowchart of a first implementation of a data processing method according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a second implementation of a data processing method according to an embodiment of the present application;
FIG. 3 is a first block diagram of a data processing system according to an embodiment of the present application;
FIG. 4 is a second block diagram of a data processing system according to an embodiment of the present application;
FIG. 5 is a diagram of a third structure of a data processing system according to an embodiment of the present application;
fig. 6 is a schematic diagram of a first structure of a data processing apparatus according to an embodiment of the present application;
fig. 7 is a second schematic structural diagram of a data processing apparatus according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of a terminal according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
Data processing is a basic link of system engineering and automatic control, and runs through various fields of social production and social life. Currently, an enterprise providing data information generally performs data processing on a service request of a client used by a user based on a server.
However, the traffic processing modules of conventional server architectures tend to be highly coupled to the clients. After the client sends a service request to the server, the service processing module acquires the service request and calls internal logic to perform data processing on the service request. Different service requests are processed by the same service processing module. The expansibility is poor, so that a developer needs to reconstruct a server framework and adjust the internal logic of the business processing module when developing according to business changes. Accordingly, the operation and maintenance work of the operation and maintenance personnel also brings difficulties, and the operation and maintenance personnel are required to be familiar with the internal architecture of the server and the related technology.
In order to explain the technical means of the present application, the following description will be given by way of specific examples.
Fig. 1 shows a schematic implementation flow diagram of a data processing method provided in an embodiment of the present application, where the method may be applied to a first service server and is applicable to a situation that the expansibility of a server architecture needs to be improved.
Specifically, the data processing method may include the following steps S101 to S104.
Step S101, obtaining a target service request.
And the service identification number carried by the target service request is the same as the service identification number corresponding to the service group where the first service server is located.
In the embodiment of the present application, a plurality of service groups are divided according to a certain rule in a server architecture, and a service server in each service group is used for executing service requests of the same type. When the client sends a service request to the server, because each service request carries a service identification number used for indicating the service type, the first service server can obtain the service request with the service identification number identical to the service identification number corresponding to the service request executed by the service group where the first service server is located.
Wherein, each service group can be divided according to the actual situation. Specifically, in some embodiments of the present application, the data content of the service request may be divided according to a correlation of the data content, or a database in which the data content of the service request is located, which is not limited in the present application.
Step S102, the target service request is analyzed to obtain the service message number carried in the target service request.
In an embodiment of the present application, the first service server may perform data processing on different data contents in service requests of the same type.
Specifically, after acquiring the service request with the same target service identification number, the first service server may analyze a data packet header of the service request to acquire a service message number corresponding to the service request. The service message number is used for identifying data processing operation required by the service request.
At this time, the first service server may allocate the service request to the target service processing module corresponding to the service message number for data processing based on the service message number.
Step S103, determining a target service processing module corresponding to the service message number in the first service server.
In an embodiment of the present application, the first service server may include a plurality of service processing modules, and each service processing module may perform a data processing operation on different data contents in the same service type. For example, the first service server may include two service processing modules, which are respectively used for executing the query daily volume data in the query market data and executing the query weekly volume data in the query market data.
After the service message number carried in the target service request is obtained through analysis, based on the service identification number corresponding to each service processing module in the first service server, the first service server may determine, according to the service message number carried in the service request, a target service processing module capable of processing the target service request in the first service server.
Step S104, the scheduling target service processing module executes a first data processing operation corresponding to the target service request to obtain a data processing result of the target service request.
In an embodiment of the application, a first service server locates a target service processing module for performing data processing on a target service request through a service message number, and the target service processing module executes a first data processing operation corresponding to the target service request to obtain a data processing result of the service request.
Specifically, the first data processing operation may be an operation such as data query or data change. In an embodiment of the present application, each service processing module may operate on a database associated with a corresponding data processing operation to implement data processing, so as to obtain a data processing result of a target service request.
In practice, the target service request may contain a plurality of data processing operations, where each data processing operation may require an operation handled by a different service processing module. In order to meet the requirement of the target service request, in some embodiments of the present application, the data processing result may be sent to the second service server, and the second service server executes a second data processing operation corresponding to the target service request according to the data processing result.
That is, after the first service server completes the current data processing operation of the service request, if the service request further includes the next data processing operation and the data processing operation that can be executed by the service processing module of the first service server does not include the next data processing operation, the data processing result may be sent to the second service server, and the second service server executes the next data processing operation corresponding to the target service request, so that the data processing operation required by the target service request can be completed, and the final data processing result is obtained.
Specifically, the first service server may send the data processing result back to the gateway, and the gateway forwards the data processing result to the second service server, or may directly send the data processing result to the second service server.
In other embodiments of the present application, the data processing result may also be fed back to the gateway, and the gateway forwards the data processing result to the client, so as to complete the response of the server to the target service request.
In the embodiment of the application, the servers are divided into different service groups, the first service server can only obtain the target service request with the same target service identification number as the service group where the first service server is located, and perform data processing on the target service request, so that the service server is coupled with the frame and the client in a loose manner, and the two service servers can be independently changed and issued under the condition that interfaces are not changed, namely, a developer can continuously add service servers for responding to the service requests with the same target service identification number in the service groups according to actual needs. Meanwhile, the first service server obtains the service message number corresponding to the target service request by analyzing the target service request, and schedules the target service processing module associated with the service message number to execute the data processing operation corresponding to the target service request, so as to obtain the data processing result of the target service request, namely, each service processing module can respectively process the service requests with the same service message number as the self-associated service message number, so that different service functions executed by each service processing module are not mixed together, and the service processing modules can be carried out concurrently without influencing each other, therefore, developers can add the service processing module in the first service server according to actual conditions, thereby solving the problem of poor expansibility of a service end framework, facilitating development and operation of a service end, reducing the cost of a service system, and saving the investment of operation and maintenance personnel, and the concurrency of the server is also improved.
It should be noted that the version number carried by the target service request may also be the same as the version number corresponding to the service group where the first service server is located.
And the target version number is a version number corresponding to the service group where the first service server is located.
That is, each service group may correspond to a target service identification number and a target version number, and each service server in the service group may process a service request whose service identification number is the same as the target service identification number and whose version number is the same as the target version number, so that the server may support multi-version coexistence and gray release. That is, in the embodiment of the present application, service requests of the same service and different versions, or service requests of different services and multiple versions, may all be simultaneously run on the server using the method provided by the present application.
In the embodiment of the present application, the service request is generally obtained from a client. Specifically, the service request sent by the client may be forwarded to the first service server in the corresponding service group through the gateway and the switch.
In order to avoid resource waste caused by processing of the same service request by other service servers in the same service group, when the target service request is a service request acquired from a message queue of the switch, the first service server may send a processed message to the switch after acquiring the target service request.
The processed message is used for preventing other service servers in the service group where the first service server is located from acquiring the target service request from the message queue of the switch.
In other embodiments of the present application, the target service request may also be obtained from another server.
Specifically, in order to ensure normal operation of the first service server, in some embodiments of the present application, the first service server may send resource occupation data of the first service server and operation data of the target service request to the scheduling server in real time, so that the scheduling server manages the first service server.
Similarly, other servers in the same service group may also send their own resource occupation data and the operation data of the service request processed by themselves to the scheduling server in real time.
Based on the monitoring and management of the scheduling server, the first service server may obtain the target service request sent by the scheduling server.
The target service request is an unfinished service request of a third service server in the service group where the first service server is located.
Specifically, after the third service server in the same service group wants the scheduling server to send its own resource occupation data, the scheduling server may determine whether to control the third service server to stop executing the service request according to the resource occupation data, for example, to close or restart the third server, so as to protect the normal operation of the service end. When the third service server is turned off or restarted, the service request currently processed by the third service server may not be completed. Therefore, the scheduling server can send the service request which is not completed by the third service server to the first service server according to the operation data of the service request of the third service server, and the first service server and the third service server are in the same service group, so that the first service server can take the service request which is not completed by the third service server as a target service request and perform data processing on the service request which is not completed by the third service server, thereby ensuring the normal operation of data processing of a server side and improving the stability of data processing.
Fig. 2 is a schematic diagram illustrating an implementation flow of a data processing method provided in an embodiment of the present application, where the method may be applied to a scheduling server and is applicable to a situation where stability of a server architecture needs to be improved.
Specifically, the data processing method may include the following steps S201 to S203.
Step S201, receiving resource occupation data of the third service server sent by the third service server.
The resource occupation data is used for representing the resource occupation condition of the third service server. When the resource occupancy rate of the third service server reaches a certain degree, the data processing speed of the third service server is reduced, the service request may be blocked in the third service server, and even the third service server may be down.
Step S202, if the resource occupation data of the third service server meets a preset load condition, sending a control instruction to the third service server.
The control instruction is used for controlling the third service server to stop executing the unfinished service request of the third service server.
In the embodiment of the present application, if the resource occupation data of the third service server does not satisfy the preset load condition, it indicates that the third service server can satisfy the requirement for executing the service request, and therefore the service request can be continuously executed by the third service server.
If the resource occupation data of the third service server meets the preset load condition, it indicates that the third service server cannot meet the requirement for executing the service request, and therefore, the scheduling server may send a control instruction to the third service server, where the control instruction is used to control the third service server to stop executing the service request that is not completed by the third service server, for example, to close or restart the third service server, so as to release the occupied resource of the third service server.
Step S203, sending the unfinished service request of the third service server to the first service server in the same service group as the third service server.
In order to ensure normal execution of the service request and to smoothly complete the service of the server, in the embodiment of the present application, when the third service server stops executing the service request that is not completed by the third service server, the service request that is not completed by the third service server may be monitored and acquired by the scheduling server.
Specifically, the third service server may send the operation data of the service request of the third service server to the scheduling server in real time, and when the third service server stops executing the service request that is not completed by the third service server, the scheduling server may determine, according to the operation data, the service request that is not completed by the third service server currently.
In an embodiment of the present application, after the scheduling server obtains an incomplete service request of the third service server, the scheduling server may send the incomplete service request to the first service server. Because the first service server and the third service server are located in the same service group, the first service server may use the service request that is not completed by the third service server as the target service request, and perform data processing on the service request that is not completed by the third service server according to the data processing method shown in fig. 1, so that the service end can normally respond to the service request.
In the embodiment of the application, the service server can dispatch the server to send the resource occupation data in real time, the dispatch server can control the service server meeting the preset load condition to stop executing the unfinished service request of the third service server by analyzing the resource occupation data, and simultaneously, the unfinished service request of the service server can be sent to other first service servers of the same service group to be finished, so that the service request cannot disappear due to the closing and restarting of the service server, and the reliability of the whole service end is ensured.
Fig. 3 is a schematic structural diagram of a data processing system provided in an embodiment of the present application, where the data processing system includes a gateway, a switch, and a first service server.
The gateway is used for receiving a target service request sent by a client, analyzing a service identification number carried in the target service request and sending the target service request to a switch of a message queue, wherein the associated service identification number is the same as the service identification number carried in the target service request; the switch is used for acquiring a target service request sent by the gateway; acquiring a related target service identification number and positioning information of a service group which is the same as the service identification number carried in the target service request; sending the target service request to a first service server in the service group according to the positioning information; the first service server is used for acquiring a target service request sent by the switch; analyzing the target service request to obtain a service message number carried in the target service request; determining a target service processing module corresponding to the service message number in the first service server; and the scheduling target service processing module executes a first data processing operation corresponding to the target service request to obtain a data processing result of the target service request.
In some embodiments of the present application, before the gateway sends the target service request to the switch, the gateway may further authenticate the service request permission of the client according to the client information; and if the client has the service request authority, analyzing a target service identification number carried in the target service request, and sending the target service request to a switch containing a message queue associated with the target service identification number.
Specifically, in the embodiment of the present application, the gateway may send information to the switch after being started, and may create a message queue and a consumer in the message queue of the switch. After the client sends the target service request, the gateway can receive the target service request sent by the client, analyze the target service request, and obtain a service identification number, a version number, a service message number and the like in the target service request. The gateway can authenticate the service request authority of the client. If the client has the service request authority, the gateway can analyze the service identification number carried in the target service request and send the target service request to the switch containing the message queue associated with the target service identification number. If the client does not have the service request permission, the gateway can return a message of request failure to the client to prompt the user of the client.
After the gateway sends the target service request to the switch containing the message queue associated with the target service identification number, the switch can receive the target service request, give the target service request to the corresponding message queue, automatically compete by consumers in the queue to obtain a request packet body, and can obtain the associated service identification number and the positioning information of the service group of the service identification number system carried in the target service request; and sending the target service request to a first service server in the service group according to the positioning information.
Then, after receiving the service request sent by the switch, the first service server can analyze the service request to obtain a service message number corresponding to the service request; and the scheduling target service processing module executes a first data processing operation corresponding to the target service request to obtain a data processing result of the target service request.
In some embodiments of the present application, the gateway, the switch, and the service server may further perform data interaction based on the version number, and the first service server may obtain a service request that has a service identification number that is the same as the service identification number corresponding to the service group where the first service server is located, and a version number that is the same as the version number corresponding to the service group where the first service server is located.
After the first service server obtains the service request identical to the target service identification number, the first service server may send a processed message to the switch, where the processed message is used to prevent other service servers in the service group where the first service server is located from obtaining the service request from the message queue of the switch.
In some embodiments of the present application, if the service request does not have the second data processing operation, the first service server may send the processing result to the switch, and the processing result is fed back to the gateway by the switch and fed back to the corresponding client by the gateway. If the service request has the second data processing operation, the first service server may send the data processing result to the second service server, and the second service server executes the second data processing operation.
In the embodiment of the application, the gateway, the switch, the service server and the service processing module in the service server are all decoupled. The service server, the frame and the client can be independently changed and issued under the condition of keeping the interface unchanged, namely, a developer can continuously increase service servers for responding to service requests with the same target service identification number in a service group according to actual needs. Meanwhile, the first service server obtains the service message number corresponding to the service request by analyzing the service request, and schedules the service processing modules associated with the service message number to perform data processing operation on the service request to obtain a data processing result of the service request, namely, each service processing module can process the service request with the same service message number as the service processing module associated with the service processing module, so that different service functions executed by each service processing module are not mixed together, and the service processing modules can be carried out concurrently without mutual influence, therefore, developers can add the service processing modules in the first service server according to actual conditions. Similarly, the gateway and the switch should be able to support multi-machine deployment and parallel expansion, effectively solving the problem of poor expansibility of the service end framework. The service servers executing the same service request can write service groups, development, management and operation and maintenance of the service end are facilitated, the development cost of a service system is reduced, the online efficiency is improved, the investment of operation and maintenance personnel is saved, and the concurrency of the service end is improved.
Further, in some embodiments of the present application, the data processing system may further include a dispatch server. Specifically, fig. 4 shows a schematic structural diagram of a data processing system provided in an embodiment of the present application, and fig. 5 shows a schematic structural diagram of the data processing system shown in fig. 4. The scheduling server may be configured to receive resource occupation data of a third service server, which is sent by a third service server located in the same service group as the first service server; if the resource occupation data of the third service server meets the preset load condition, sending a control instruction to the third service server to control the third service server to stop executing the unfinished service request of the third service server; and sending the service request which is not completed by the third service server to the first service server.
It should be noted that the service servers in each service group can send their own resource occupation data and operation data of the service request to the scheduling server in real time, and the scheduling server manages the server according to the resource occupation data and the operation data of the service request.
In the embodiment of the application, the scheduling server in the data processing system can manage the service server according to the resource occupation data, and for the unfinished service request of the service server, the service request can be sent to other service servers in the service group where the service server is located, so that the service request can be finished normally, and the reliability of the service end is ensured.
It should be noted that, for simplicity of description, the foregoing method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts, as some steps may, in accordance with the present application, occur in other orders.
Fig. 6 is a schematic structural diagram of a data processing apparatus 600 according to an embodiment of the present application, where the data processing apparatus 600 is configured on a first service server.
Wherein the data processing apparatus 600 may comprise: a service request acquisition unit 601, a service request parsing unit 602, a module determination unit 603, and a data processing unit 604.
A service request obtaining unit 601, configured to obtain a target service request; the service identification number carried by the target service request is the same as the service identification number corresponding to the service group where the first service server is located;
a service request analyzing unit 602, configured to analyze the target service request to obtain a service message number carried in the target service request;
a module determining unit 603, configured to determine a target service processing module corresponding to the service message number in the first service server;
a data processing unit 604, configured to schedule the target service processing module to execute a first data processing operation corresponding to the target service request, so as to obtain a data processing result of the target service request.
In some embodiments of the present application, the data processing unit 604 may be further specifically configured to: and if the service request has the next data processing operation different from the target service identification number, sending the service request and the data processing result to a second service server, and executing the next data processing operation on the service request by the second service server.
In some embodiments of the present application, the service request obtaining unit 601 may be further specifically configured to: acquiring a service request sent by a scheduling server; the service request is an unfinished service request when a third service server in the service group where the first service server is located is closed or restarted.
In some embodiments of the present application, the data processing apparatus 600 may further include a monitoring unit, configured to send, to the scheduling server, resource occupation data of the first service server and operation data of the service request in real time.
In some embodiments of the present application, if the service request is a service request acquired from a message queue of a switch, the service request acquiring unit 601 may be further specifically configured to: and sending a processed message to the switch, wherein the processed message is used for preventing other service servers in the service group where the first service server is located from acquiring the service request from a message queue of the switch.
In some embodiments of the present application, the service request obtaining unit 601 may be further specifically configured to: acquiring a service request which is the same as the target service identification number and the target version number; and the target version number is a version number corresponding to the service group where the first service server is located.
It should be noted that, for convenience and simplicity of description, the specific working process of the data processing apparatus 600 may refer to a corresponding process of the method described in fig. 1, and is not described herein again.
Fig. 7 is a schematic structural diagram of a data processing apparatus 700 according to an embodiment of the present application, where the data processing apparatus 700 is configured on a scheduling server.
The data processing apparatus 700 may include: a resource data receiving unit 701, a control instruction transmitting unit 702, and a service request transmitting unit 703.
A resource data receiving unit 701, configured to receive resource occupation data of a third service server sent by the third service server;
a control instruction sending unit 702, configured to send a control instruction to the third service server if the resource occupation data of the third service server meets a preset load condition, where the control instruction is used to control the third service server to stop executing a service request that is not completed by the third service server;
a service request sending unit 703, configured to send the service request that is not completed by the third service server to a first service server in the same service group as the third service server, where the first service server performs data processing on the service request that is not completed by the third service server according to the data processing method shown in fig. 1.
It should be noted that, for convenience and simplicity of description, the specific working process of the data processing apparatus 700 may refer to a corresponding process of the method described in fig. 2, and is not described herein again.
Fig. 8 is a schematic diagram of a terminal according to an embodiment of the present application. The terminal may be the first server or the dispatch server.
Among them, the terminal 8 may include: a processor 80, a memory 81 and a computer program 82, such as a data processing program, stored in said memory 81 and operable on said processor 80.
When the terminal is a first server, the processor 80 executes the computer program 82 to implement the steps in the above-mentioned data processing method embodiments, for example, steps S101 to S104 shown in fig. 1. Alternatively, the processor 80, when executing the computer program 82, implements the functions of the modules/units in the above-mentioned device embodiments, such as the service request obtaining unit 601, the service request parsing unit 602, the module determining unit 603, and the data processing unit 604 shown in fig. 6.
When the terminal is a dispatch server, the processor 80 executes the computer program 82 to implement the steps in the above-described embodiments of the data processing method, for example, steps S201 to S203 shown in fig. 2. Alternatively, the processor 80, when executing the computer program 82, implements the functions of the modules/units in the above-mentioned device embodiments, such as the resource data receiving unit 701, the control instruction sending unit 702, and the service request sending unit 703 shown in fig. 7.
The computer program may be divided into one or more modules/units, which are stored in the memory 81 and executed by the processor 80 to accomplish the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution of the computer program in the terminal.
For example, the computer program may be divided into: the system comprises a service request acquisition unit, a service request analysis unit, a module determination unit and a data processing unit. The specific functions of each unit are as follows: a service request acquiring unit, configured to acquire a target service request; the service identification number carried by the target service request is the same as the service identification number corresponding to the service group where the first service server is located; a service request analyzing unit, configured to analyze the target service request to obtain a service message number carried in the target service request; a module determining unit, configured to determine a target service processing module corresponding to the service message number in the first service server; and the data processing unit is used for scheduling the target service processing module to execute a first data processing operation corresponding to the target service request to obtain a data processing result of the target service request.
For example, the computer program may be divided into: the system comprises a resource data receiving unit, a control instruction sending unit and a service request sending unit. The specific functions of each unit are as follows: a resource data receiving unit, configured to receive resource occupation data of a third service server sent by the third service server; a control instruction sending unit, configured to send a control instruction to the third service server if the resource occupation data of the third service server meets a preset load condition, where the control instruction is used to control the third service server to stop executing a service request that is not completed by the third service server; a service request sending unit, configured to send a service request that is not completed by the third service server to a first service server in the same service group as the third service server, where the first service server performs data processing on the service request that is not completed by the third service server according to the data processing method shown in fig. 1.
The terminal may include, but is not limited to, a processor 80, a memory 81. Those skilled in the art will appreciate that fig. 8 is merely an example of a terminal and is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or different components, e.g., the terminal may also include input-output devices, network access devices, buses, etc.
The Processor 80 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 81 may be an internal storage unit of the terminal, such as a hard disk or a memory of the terminal. The memory 81 may also be an external storage device of the terminal, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are equipped on the terminal. Further, the memory 81 may also include both an internal storage unit and an external storage device of the terminal. The memory 81 is used for storing the computer program and other programs and data required by the terminal. The memory 81 may also be used to temporarily store data that has been output or is to be output.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow in the method of the embodiments described above can be realized by a computer program, which can be stored in a computer-readable storage medium and can realize the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A data processing method is applied to a first service server, and the data processing method comprises the following steps:
acquiring a target service request; the service identification number carried by the target service request is the same as the service identification number corresponding to the service group where the first service server is located;
analyzing the target service request to obtain a service message number carried in the target service request;
determining a target service processing module corresponding to the service message number in the first service server;
and scheduling the target service processing module to execute a first data processing operation corresponding to the target service request to obtain a data processing result of the target service request.
2. The data processing method of claim 1, after obtaining the data processing result of the service request, comprising:
and sending the data processing result to a second service server, and executing a second data processing operation corresponding to the target service request by the second service server according to the data processing result.
3. The data processing method of claim 1 or 2, wherein the obtaining a target service request comprises:
acquiring a target service request sent by a scheduling server; the target service request is a service request which is not completed by a third service server in the service group where the first service server is located.
4. A data processing method according to claim 3, characterized in that the method further comprises:
and transmitting the resource occupation data of the first service server and the operation data of the target service request to the scheduling server in real time.
5. The data processing method according to claim 1 or 2, wherein if the target service request is a service request obtained from a message queue of a switch, after obtaining the target service request, the method comprises:
and sending a processed message to the switch, wherein the processed message is used for preventing other service servers in the service group where the first service server is located from acquiring the target service request from a message queue of the switch.
6. The data processing method according to claim 1 or 2, wherein the target service request carries a version number that is the same as a version number corresponding to the service group in which the first service server is located.
7. A data processing method is applied to a scheduling server, and comprises the following steps:
receiving resource occupation data of a third service server sent by the third service server;
if the resource occupation data of the third service server meets a preset load condition, sending a control instruction to the third service server, wherein the control instruction is used for controlling the third service server to stop executing the unfinished service request of the third service server;
and sending the service request which is not completed by the third service server to a first service server which is positioned in the same service group with the third service server, and performing data processing on the service request which is not completed by the third service server by the first service server according to the data processing method of any one of claims 1 to 6.
8. A data processing system, comprising a gateway, a switch, and a first service server;
the gateway is used for receiving a target service request sent by a client, analyzing a service identification number carried in the target service request and sending the target service request to a switch of a message queue with the associated service identification number being the same as the service identification number carried in the target service request;
the switch is used for acquiring the target service request sent by the gateway; acquiring a related target service identification number and positioning information of a service group which is the same as the service identification number carried in the target service request; sending the target service request to a first service server in the service group according to the positioning information;
the first service server is used for acquiring the target service request sent by the switch; analyzing the target service request to obtain a service message number carried in the target service request; determining a target service processing module corresponding to the service message number in the first service server; and scheduling the target service processing module to execute a first data processing operation corresponding to the target service request to obtain a data processing result of the target service request.
9. A terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 6 when executing the computer program or implements the steps of the method according to claim 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 6, or which, when being executed by a processor, carries out the steps of the method according to claim 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110440483.7A CN113515369B (en) | 2021-04-23 | 2021-04-23 | Data processing method, system, terminal and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110440483.7A CN113515369B (en) | 2021-04-23 | 2021-04-23 | Data processing method, system, terminal and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113515369A true CN113515369A (en) | 2021-10-19 |
| CN113515369B CN113515369B (en) | 2022-03-29 |
Family
ID=78061181
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110440483.7A Active CN113515369B (en) | 2021-04-23 | 2021-04-23 | Data processing method, system, terminal and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113515369B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114710311A (en) * | 2022-02-11 | 2022-07-05 | 浙江高信技术股份有限公司 | Multi-project message management method and system |
| CN115378801A (en) * | 2022-08-10 | 2022-11-22 | 福州六察网络科技有限公司 | Multi-server communication method and terminal |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR3007547A1 (en) * | 2013-06-19 | 2014-12-26 | Bull Sas | METHOD FOR TRANSFERRING DATA IN A DYNAMIC ENVIRONMENT |
| CN104852964A (en) * | 2015-04-10 | 2015-08-19 | 北京金和软件股份有限公司 | Multifunctional server scheduling method |
| CN105847425A (en) * | 2016-05-17 | 2016-08-10 | 乐视控股（北京）有限公司 | Service request processing method and device |
| CN106330769A (en) * | 2016-11-15 | 2017-01-11 | 腾讯科技（深圳）有限公司 | Business processing method and server |
| CN106411971A (en) * | 2015-07-29 | 2017-02-15 | 腾讯科技（深圳）有限公司 | Load regulation method and device |
| CN108965380A (en) * | 2018-05-31 | 2018-12-07 | 平安医疗科技有限公司 | Service request processing method, device, computer equipment and storage medium |
| CN110298769A (en) * | 2019-06-13 | 2019-10-01 | 浙江口碑网络技术有限公司 | The processing method for service request of ordering, apparatus and system |
| CN110581893A (en) * | 2019-09-18 | 2019-12-17 | 西安未来国际信息股份有限公司 | data transmission method and device, routing equipment, server and storage medium |
| CN110673959A (en) * | 2019-10-14 | 2020-01-10 | 京东数字科技控股有限公司 | System, method and apparatus for processing tasks |
-
2021
- 2021-04-23 CN CN202110440483.7A patent/CN113515369B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR3007547A1 (en) * | 2013-06-19 | 2014-12-26 | Bull Sas | METHOD FOR TRANSFERRING DATA IN A DYNAMIC ENVIRONMENT |
| CN104852964A (en) * | 2015-04-10 | 2015-08-19 | 北京金和软件股份有限公司 | Multifunctional server scheduling method |
| CN106411971A (en) * | 2015-07-29 | 2017-02-15 | 腾讯科技（深圳）有限公司 | Load regulation method and device |
| CN105847425A (en) * | 2016-05-17 | 2016-08-10 | 乐视控股（北京）有限公司 | Service request processing method and device |
| CN106330769A (en) * | 2016-11-15 | 2017-01-11 | 腾讯科技（深圳）有限公司 | Business processing method and server |
| CN108965380A (en) * | 2018-05-31 | 2018-12-07 | 平安医疗科技有限公司 | Service request processing method, device, computer equipment and storage medium |
| CN110298769A (en) * | 2019-06-13 | 2019-10-01 | 浙江口碑网络技术有限公司 | The processing method for service request of ordering, apparatus and system |
| CN110581893A (en) * | 2019-09-18 | 2019-12-17 | 西安未来国际信息股份有限公司 | data transmission method and device, routing equipment, server and storage medium |
| CN110673959A (en) * | 2019-10-14 | 2020-01-10 | 京东数字科技控股有限公司 | System, method and apparatus for processing tasks |
Non-Patent Citations (2)
| Title |
|---|
| BOYANG YU 等: "Optimize the Server Provisioning and Request Dispatching in Distributed Memory Cache Services", 《IEEE TRANSACTIONS ON CLOUD COMPUTING》 * |
| 徐胜超 等: "基于协同服务器组的志愿者计算环境的构造", 《计算机研究与发展》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114710311A (en) * | 2022-02-11 | 2022-07-05 | 浙江高信技术股份有限公司 | Multi-project message management method and system |
| CN114710311B (en) * | 2022-02-11 | 2024-01-02 | 浙江高信技术股份有限公司 | Multi-project message management method and system |
| CN115378801A (en) * | 2022-08-10 | 2022-11-22 | 福州六察网络科技有限公司 | Multi-server communication method and terminal |
| CN115378801B (en) * | 2022-08-10 | 2024-01-30 | 福州六察网络科技有限公司 | Multi-server communication method and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113515369B (en) | 2022-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108874558B (en) | Message subscription method of distributed transaction, electronic device and readable storage medium | |
| CN112073269B (en) | Block chain network testing method, device, server and storage medium | |
| CN113515369B (en) | Data processing method, system, terminal and storage medium | |
| CN110750592B (en) | Data synchronization method, device and terminal equipment | |
| CN111061798B (en) | Configurable data transmission and monitoring method, equipment and medium | |
| CN102629207A (en) | Method for running instance time-sharing update under multi-tenant condition | |
| CN108763482B (en) | Method and device for scheduling data analysis process and server cluster | |
| CN111381940B (en) | Distributed data processing method and device | |
| CN108616668B (en) | Method for monitoring dialing task, electronic device and computer readable storage medium | |
| CN111694677A (en) | Message queue management method, device, terminal and computer-readable storage medium | |
| CN112286664A (en) | Task scheduling method and device, computer equipment and readable storage medium | |
| CN111091439B (en) | Order-related marketing campaign realization method, system, equipment and storage medium | |
| CN111459629A (en) | Azkaban-based project operation method and device and terminal equipment | |
| CN116456496B (en) | Resource scheduling method, storage medium and electronic equipment | |
| CN108521524B (en) | Agent collaborative task management method and device, computer equipment and storage medium | |
| CN112148506A (en) | Message processing method, device, platform and storage medium | |
| CN109308219B (en) | Task processing method and device and distributed computer system | |
| CN115470206A (en) | MES system docking data interaction method and system, computer equipment and storage medium | |
| CN107464088B (en) | Service processing system and method based on message queue | |
| CN115617480A (en) | Task scheduling method, device and system and storage medium | |
| CN115185841A (en) | System reconfiguration test method, device, equipment and storage medium | |
| CN113612832A (en) | Streaming data distribution method and system | |
| CN113065927A (en) | Account checking method and device, electronic equipment and computer readable storage medium | |
| CN114648410A (en) | Stock staring method, apparatus, system, device and medium | |
| CN113760315A (en) | Method and device for testing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |