CN111597057B - Data processing method, device and computer readable storage medium - Google Patents
Data processing method, device and computer readable storage medium Download PDFInfo
- Publication number
- CN111597057B CN111597057B CN202010270735.1A CN202010270735A CN111597057B CN 111597057 B CN111597057 B CN 111597057B CN 202010270735 A CN202010270735 A CN 202010270735A CN 111597057 B CN111597057 B CN 111597057B
- Authority
- CN
- China
- Prior art keywords
- packet
- service
- service command
- identity
- packets
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/547—Remote procedure calls [RPC]; Web services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/541—Client-server
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/50—Reducing energy consumption in communication networks in wire-line communication networks, e.g. low power modes or reduced link rate
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a data processing method, a device and a computer readable storage medium, wherein the method comprises the following steps: receiving at least one service parameter input by a user, determining a service command packet corresponding to each service parameter according to the at least one service parameter, transmitting each service command packet to a server by using one service command packet carrying a unique identity, receiving original response packets corresponding to each service command packet and returned by the server, wherein the original response packets corresponding to each service command packet and the service command packet carry the same identity, and determining complete response packets corresponding to each service command packet according to the identity, the total number of packet fragments and the current fragment number carried by each original response packet.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a data processing method, apparatus, and computer readable storage medium.
Background
In the existing data processing system, when a user initiates a service request, the situation that the same user initiates the same service request for a plurality of times is easy to occur, at this moment, a server receives the service request for the same user for a plurality of times, and generates a plurality of response data for the plurality of service requests respectively, at this moment, if the plurality of response data are too large, the plurality of response data need to be processed in a slicing way respectively and then sent to the data processing system, and after the data processing system receives the response data, the data processing system merges a plurality of original response packets respectively contained in the plurality of response data to obtain a complete response packet, thereby obtaining the complete response data.
However, the existing original response inclusion combination mode uses the service identifier as the combination basis, so that after the data processing system receives the original response packet returned by the server, the combination of the original response packet based on the service identifier is inaccurate, and the service abnormality is easy to cause.
Disclosure of Invention
In view of the foregoing, embodiments of the present invention are directed to a data processing method, apparatus, and computer-readable storage medium that overcome, or at least partially solve, the foregoing problems.
To solve the above problems, an embodiment of the present invention discloses a data processing method, which is applied to a data processing system, where the data processing system is communicatively connected to a server, and the method includes:
receiving at least one business parameter input by a user;
determining service command packets corresponding to the service parameters according to at least one service parameter, wherein one service command packet carries a unique identity;
sending each service command packet to the server;
receiving original response packets corresponding to the service command packets respectively returned by the server, wherein the corresponding original response packets and the service command packets carry the same identity;
And determining the complete response packet corresponding to each service command packet according to the identity mark carried by each original response packet, the total number of packet fragments and the current fragment number.
Optionally, the service parameters include service identifiers and service data, and determining, according to at least one service parameter, a service command packet corresponding to each service parameter includes:
assigning a value to each service identifier to generate an identity identifier corresponding to each service identifier, wherein each identity identifier has uniqueness;
and packaging the service data according to the identity and a preset packaging rule to generate a service command packet corresponding to each service parameter.
Optionally, before determining the complete response packet corresponding to each service command packet according to the identity identifier carried by each original response packet, the total number of packet fragments and the current fragment number, the method further includes:
judging whether the complete response packet corresponding to each service command packet is fragmented or not according to the total number of packet fragments carried by each original response packet;
the determining the complete response packet corresponding to each service command packet according to the identity, the total number of packet fragments and the current fragment number carried by each original response packet includes:
Under the condition that the complete response packet corresponding to the service command packet is processed in a slicing way, taking the original response packet corresponding to the service command packet as the complete response packet corresponding to the service command packet;
and under the condition that the data packet corresponding to the service command packet is not fragmented, sequencing and merging the original response packets according to the current fragment numbers of the original response packets and according to all the original response packets corresponding to the same identity mark to obtain a complete response packet corresponding to the service command packet.
Optionally, before all original response packets corresponding to the same identity are sequenced and combined according to the current fragment number of each original response packet to obtain a complete response packet corresponding to the service command packet, the method further includes:
under the condition that the complete response packet corresponding to the service command packet is processed in a fragmented way, establishing an association relation between the identity of the service command packet and a response packet storage queue;
storing all original response packets corresponding to the identity mark into the response packet storage queue;
judging whether the number of original response packets in the response packet storage queue is equal to the total number of packet fragments;
And under the condition that the number of the original response packets in the response packet storage queue is equal to the total number of the packet fragments, taking the original response packets in the response packet storage queue as all the original response packets corresponding to the same identity.
Optionally, after each service command packet is sent to the server, the method further includes:
judging whether a complete response packet corresponding to the service command packet can be locally acquired according to the identity of the service command packet;
acquiring response timeout time corresponding to the service command packet under the condition that the complete response packet is not acquired;
judging whether the response timeout time is greater than a preset timeout time;
and returning to the step under the condition that the response timeout time is smaller than the preset timeout time: judging whether a complete response packet corresponding to the service command packet can be obtained locally according to the identity of the service command packet until the response timeout time is longer than the preset timeout time or until the complete response packet corresponding to the service command packet is obtained locally according to the identity of the service command packet.
The embodiment of the invention also discloses a data processing device which is applied to a data processing system, wherein the data processing system is in communication connection with a server, and the device comprises:
The first receiving module is used for receiving at least one business parameter input by a user;
the first determining module is used for determining service command packets corresponding to the service parameters according to at least one service parameter, wherein one service command packet carries a unique identity;
the sending module is used for sending each service command packet to the server;
the second receiving module is used for receiving original response packets corresponding to the service command packets respectively returned by the server, wherein the original response packets and the service command packets which correspond to each other carry the same identity;
and the second determining module is used for determining the complete response packet corresponding to each service command packet according to the identity mark carried by each original response packet, the total number of packet fragments and the current fragment number.
Optionally, the service parameter includes a service identifier and service data, and the first determining module includes:
the first generation sub-module is used for assigning a value to each service identifier to generate an identity identifier corresponding to each service identifier, and each identity identifier has uniqueness;
and the second generation sub-module is used for packaging the service data according to the identity mark and a preset packaging rule to generate a service command packet corresponding to the service parameter.
Optionally, the apparatus further comprises:
the first judging module is used for judging whether the complete response packet corresponding to each service command packet is fragmented or not according to the total number of packet fragments carried by each original response packet;
the second determining module includes:
the first obtaining submodule is used for taking an original response packet corresponding to the service command packet as a complete response packet corresponding to the service command packet under the condition that the complete response packet corresponding to the service command packet is processed in a slicing way;
and the second obtaining submodule is used for sequencing and combining the original response packets according to the current fragment numbers of the original response packets and the current fragment numbers of the original response packets according to all the original response packets corresponding to the same identity under the condition that the data packets corresponding to the service command packets are not fragmented, so as to obtain complete response packets corresponding to the service command packets.
Optionally, the apparatus further comprises:
the creation module is used for creating the association relation between the identity of the service command packet and the response packet storage queue under the condition that the complete response packet corresponding to the service command packet is processed in a fragmented manner;
the storage module is used for storing all original response packets corresponding to the identity mark into the response packet storage queue;
The second judging module is used for judging whether the number of the original response packets in the response packet storage queue is equal to the total number of the packet fragments;
and the third determining module is used for taking the original response packets in the response packet storage queue as all original response packets corresponding to the same identity mark under the condition that the number of the original response packets in the response packet storage queue is equal to the total number of the packet fragments.
Optionally, the apparatus further comprises:
the third judging module is used for judging whether the complete response packet corresponding to the service command packet can be locally acquired according to the identity of the service command packet;
the obtaining module is used for obtaining response timeout time corresponding to the service command packet under the condition that the complete response packet is not obtained;
a fourth judging module, configured to judge whether the response timeout time is greater than a preset timeout duration;
the returning module is configured to return to the step if the response timeout time is less than the preset timeout duration: judging whether a complete response packet corresponding to the service command packet can be obtained locally according to the identity of the service command packet until the response timeout time is longer than the preset timeout time or until the complete response packet corresponding to the service command packet is obtained locally according to the identity of the service command packet.
The embodiment of the invention also discloses a data processing device, which comprises:
one or more processors; and
one or more computer-readable media having instructions stored thereon, which when executed by the one or more processors, cause the apparatus to perform a data processing method according to any of the embodiments of the present invention.
The embodiment of the invention also discloses a computer readable storage medium, and a stored computer program causes a processor to execute the data processing method according to the embodiment of the invention.
The embodiment of the invention has the following advantages:
in the data processing method provided by the embodiment of the invention, by receiving at least one service parameter input by a user, determining a service command packet corresponding to each service parameter according to at least one service parameter, wherein one service command packet carries a unique identity, sending each service command packet to a server, receiving original response packets corresponding to each service command packet and returned by the server, respectively, wherein the original response packets corresponding to each service command packet carry the same identity, and determining complete response packets corresponding to each service command packet according to the identity carried by each original response packet, the total number of packet fragments and the current fragment number.
Drawings
FIG. 1 is a flow chart illustrating steps of a data processing method according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating steps of another data processing method according to an embodiment of the present invention;
FIG. 3 is a block diagram illustrating portions of a data processing system according to one embodiment of the present invention;
FIG. 4 is a functional block diagram of portions of a data processing system according to one embodiment of the present invention;
fig. 5 is a block diagram of a data processing apparatus according to an embodiment of the present invention.
Detailed Description
In order that the above-recited objects, features and advantages of the present invention will become more readily apparent, a more particular description of the invention will be rendered by reference to the appended drawings and appended detailed description.
When a user initiates a service request, if two identical service requests are continuously initiated, namely, an original response packet corresponding to a first service request is not received, a second service request is sent again, at this time, service identifiers carried by the front service request and the rear service request are identical, and the existing original response inclusion mode is based on the service identifiers, so that the data processing system faces to a plurality of original response packets corresponding to the received front service request and the rear service request, and when the original response packets are merged, the original response packets corresponding to the front service request and the rear service request cannot be distinguished correctly according to the service identifiers, and the situation that merging errors easily occur, namely, the original response packet of the second service request is wrongly taken as the original response packet of the first service request to be merged, abnormal data is generated, and service abnormality is caused.
Referring to fig. 1, fig. 1 shows a flowchart of steps of a data processing method according to an embodiment of the present invention, where the method may be applied to a data processing system, and the data processing system is communicatively connected to a server, and may specifically include the following steps:
step S11, at least one business parameter input by a user is received.
In this embodiment, the data processing system may be a system developed based on a server, and the data processing system may be connected to the server through a TCP socket, and log in to the server using an account registered in the server, and then may acquire response data generated by the server. In addition, the data processing system can receive the business parameters input by the user, so as to process the business of the user.
The service parameters may include a service identifier and service data, where the service identifier is used to uniquely identify a type of a service request initiated by the user, for example, the service identifier may identify that the service initiated by the user is a download service, a conference service, or a video playing service, etc.; the service data is operation data for a service request, and may be an object for which a user performs the operation, for example, a download object for downloading a service, a play object for a participant or a video play service for a conference service, and the like.
The type of the service request of the user may be various service requests in the network, such as a file downloading service, a conference service, a video playing service, etc., and the service of the user is not particularly limited in this embodiment.
Taking the user service identifier as a file downloading service as an example, the user may perform misoperation when operating, and click the same downloading key twice in a short time, at this time, the data processing system may receive two service parameters input by the user, and because the two downloading services are the same downloading key clicked by the user, the service identifiers corresponding to the two service parameters are the same, or, when the user performs the downloading service, the first downloading service selects the 1 st and the 2 nd file downloading, then starts the second downloading service, and the second downloading service selects the 3 rd and the 4 th file downloading, at this time, the data processing system may receive the two service parameters input by the user, and similarly, because the two downloading services are the same downloading key clicked by the user, the service identifiers corresponding to the two service parameters are the same.
Step S12, determining service command packets corresponding to the service parameters according to at least one service parameter, wherein one service command packet carries a unique identity.
In this embodiment, after receiving the service parameters, the data processing system may process each service parameter to generate service command packets corresponding to each service parameter, where each service command packet carries a unique identity. The service command packet is a data packet for the server to parse and execute and generate a complete response packet.
In one embodiment, the step S22 may specifically include the following steps:
step S121, assigning a value to each service identifier, and generating an identity identifier corresponding to each service identifier, where each identity identifier has uniqueness.
Step S122, according to the identity and preset encapsulation rules, encapsulating each service data to generate service command packets corresponding to each service parameter.
In this embodiment, after receiving the service parameter, the data processing system may obtain the service identifier in the service parameter, and assign different values to each service identifier, where the service identifier after assignment is an identity identifier corresponding to the service identifier, so that each identity identifier generated after processing by the data processing system has uniqueness.
In one embodiment, when the data processing system assigns different values to each service identifier, a numerical accumulation rule may be specifically adopted to perform assignment, and the original service identifier is not deleted in the assignment process. For example, the data processing system receives 3 service identifiers A, A, B successively, where the received first service identifier a is assigned 1, the second service identifier a is assigned 2, and the third service identifier B is assigned 3, where the assigned service identifiers are A1, A2, and B3, respectively. In this embodiment, the value is assigned by the rule of numerical accumulation, so that each service identifier can be guaranteed to have uniqueness, and the original service identifier cannot be deleted in the assignment process, so that the server can conveniently identify the type of the service request.
After each service identifier is assigned, service data belonging to the same service parameter can be encapsulated according to the identity identifier and a preset encapsulation rule, so as to generate a service command packet corresponding to each service parameter, and any suitable prior art can be referred to for a specific encapsulation process of the data packet, which is not described herein.
In one embodiment, the identity identifier may be encapsulated in the outermost layer of the service command packet according to a preset encapsulation rule, so that the server or the data processing system may parse the data packet faster to obtain the identity identifier without parsing the inner layer of the service command packet.
And step S13, each service command packet is sent to the server.
In this embodiment, the data processing system may create a packet queue locally, and after generating the service command packet, the data processing system may place the service command packet in the packet queue, and wait to send the service command packet in the packet queue.
Step S14, receiving original response packets corresponding to the service command packets respectively returned by the server, wherein the corresponding original response packets and the service command packets carry the same identity.
In this embodiment, the original response packet is a response packet in the original state received by the data processing system, that is, a response packet in which the data processing system has not performed the merging process.
After receiving each service command packet sent by the data processing system, the server can analyze each service command packet to obtain an identity and service data corresponding to each service command packet, perform corresponding processing to generate response data corresponding to each service command packet, package each response data according to a preset package rule, and send the response data to the data processing system, so that the data processing system can start a thread to receive the original response packet corresponding to each service command packet returned by the server in real time. Because the server can obtain the identity and the service data corresponding to each service command packet, the server can add the corresponding identity in the response packet when packaging the response packet, so that the original response packet correspondingly generated by the server and the service command packet received by the server can carry the same identity.
For example, taking a download task as an example, after receiving the service command packet, the server can parse the service command packet to obtain an identity (download command and assignment) and a download object, obtain download data corresponding to the download object from the database as response data, package the identity (download command and assignment) and the response data into an original response packet, and send the original response packet to the data processing system.
And step S15, determining complete response packets corresponding to the service command packets respectively according to the identity marks carried by the original response packets, the total number of the packet fragments and the current fragment number.
In this embodiment, because of the packet size limitation of the server, there are two possible cases where the server generates the response data according to the service command packet, and one case is that the response data exceeds the packet size limitation, where the response data needs to be divided into a plurality of packets to be sent, that is, the server performs the fragmentation processing on the generated response data, generates a plurality of fragmented packets, and sends the fragmented packets to the data processing system; alternatively, the response data does not exceed the packet size limit, and the response data is directly transmitted in one response packet without the need for fragmentation processing. In either case, however, the server will carry the identity, the total number of packet fragments, and the current fragment number when encapsulating the data. If the server does not need to carry out slicing processing when packaging the response data, the total number of packet slicing of the packaged response packet corresponding to the identity is 1, and the current slicing number is 1; if the server needs to process the fragments when packaging the response data, the response data is divided into a plurality of data packets when packaged, the total number of the fragments of the response packets corresponding to the identity is what, and in addition, the current fragment number of the response packets corresponding to the identity is determined according to the actual packaging sequence.
After receiving the original response packets returned by the server, the data processing system can analyze each original response packet to obtain the identity information, the total number of packet fragments and the current fragment number carried by each original response packet, and obtain the complete response packet corresponding to each service command packet according to the identity information, the total number of packet fragments and the current fragment number carried by each original response packet.
In this embodiment, by receiving at least one service parameter input by a user, determining a service command packet corresponding to each service parameter according to at least one service parameter, one service command packet carries a unique identity, sending each service command packet to a server, receiving an original response packet returned by the server and corresponding to each service command packet, where the original response packet and the service command packet corresponding to each other carry the same identity, and determining a complete response packet corresponding to each service command packet according to the identity carried by each original response packet, the total number of packet fragments and the current fragment number.
In combination with the above embodiment, considering that when the server returns the complete response packet, some response data will be fragmented and some response data will not be fragmented, so, in order to accurately obtain the complete response packet corresponding to the service command packet, referring to fig. 2, fig. 2 is another data processing method provided in an embodiment of the present application, as shown in fig. 2, before step S15, the data processing method of the present application may further include the following steps:
step S104, judging whether the complete response packet corresponding to each service command packet is fragmented or not according to the total number of the fragmented packets carried by each original response packet.
Accordingly, the step S15 may specifically include the following steps:
step S151, taking the original response packet corresponding to the service command packet as the complete response packet corresponding to the service command packet when the complete response packet corresponding to the service command packet has been fragmented.
Step S152, in the case that the data packet corresponding to the service command packet is not fragmented, sorting and merging the original response packets according to the current fragmentation numbers of the original response packets and according to all the original response packets corresponding to the same identity, so as to obtain a complete response packet corresponding to the service command packet.
In this embodiment, the data processing system may determine, according to the total number of fragments of the packets carried by each original response packet, whether the complete response packet corresponding to each service command packet has been processed by fragmentation.
For example, if the total number of the fragments of the packets carried by the original response packet is 1, it is indicated that the complete response packet corresponding to the service command packet identified by the identity in the original response packet is not fragmented, and if the total number of the fragments of the packets carried by the original response packet is not 1, it is indicated that the original response packet is fragmented.
If the complete response packet corresponding to the service command packet is not processed by the fragments, the original response packet does not need to be combined, and the original response packet can be directly used as the complete response packet corresponding to the service command packet; if the complete response packet corresponding to the service command packet is already fragmented, the original response packet needs to be merged. Specifically, the data processing system is required to receive all original response packets corresponding to the same identity, the data in all original response packets form response data of the service command packet, then all original response packets are ordered according to the current fragment number of each original response packet, finally all original response packets after being ordered are combined, and finally the complete response packet corresponding to the service command packet is obtained.
In this embodiment, whether the complete response packet corresponding to each service command packet has been fragmented is determined, so that the complete response packet corresponding to each service command packet can be conveniently and accurately obtained.
In one embodiment, after the complete response packet corresponding to the service command packet is obtained, the service command packet may be temporarily stored locally in a key-value pair manner, specifically, the identity is used as a key, and the complete response packet is used as a value.
In one embodiment, to facilitate management of each original response packet received by the data processing system, the data processing method of the present application may further include, before step S152, the following steps:
step S1041, when the complete response packet corresponding to the service command packet has been fragmented, establishes an association relationship between the identity of the service command packet and the response packet storage queue.
Step S1042, storing all original response packets corresponding to the identity into the response packet storage queue.
Step S1043, determining whether the number of original response packets in the response packet storage queue is equal to the total number of packet fragments.
Step S1044, taking the original response packets in the response packet storage queue as all the original response packets corresponding to the same identity identifier when the number of the original response packets in the response packet storage queue is equal to the total number of the packet fragments.
In this embodiment, if the data processing system determines that the complete response packet corresponding to the service command packet has been processed by fragmentation, at this time, the data processing system may create a response packet storage queue, and establish an association relationship between an identity identifier of the service command packet and the response packet storage queue, to store original response packets corresponding to each service command packet, specifically, data in the response storage queue may be stored in a key value pair manner, specifically, may use the identity identifier as a key, use a set corresponding to the identity identifier as a value, and store all original response packets corresponding to the identity identifier in the set corresponding to the identity identifier.
After storing the corresponding original data packets according to the identities, the data processing system may determine whether the number of original response packets corresponding to each identity in the response packet storage queue is equal to the total number of packet fragments, if the number of original response packets in the response packet storage queue is equal to the total number of packet fragments, it indicates that all the original data packets corresponding to the identity have been received, at this time, all the original response packets corresponding to the identity in the response packet storage queue may be used as all the original response packets corresponding to the same identity, and then the above step S152 is continuously executed.
In this embodiment, the association relationship between the identity of the service command packet and the response packet storage queue is established to store each original response packet, so that each original response packet is convenient to manage, and the total number of the original response packets corresponding to each identity is convenient to count, so that whether the original response packets with the same identity are received or not is judged, and the obtaining efficiency of the complete response packet is improved.
In one embodiment, considering that the data processing system may establish multiple TCP socket connections with the server, so as to provide access to multiple users, where multiple users may send service parameters to the data processing system simultaneously, in some extreme cases, for example, two users send two service parameters in front and back very close time intervals, which may cause the data processing system to assign the same value when assigning the service identifier included in the service parameters, where the complete response packet corresponding to the service command packet cannot be obtained accurately according to the identity identifier.
In addition, considering that there may be a certain normal delay in the actual process of acquiring the complete response packet according to the service command packet and a certain time is required for transmitting the complete data packet, in order to improve the accuracy of acquiring the complete response packet by the data processing system, after step S13, the data processing method of the embodiment of the present application may further include the following steps:
step S131, judging whether the complete response packet corresponding to the service command packet can be obtained locally according to the identity of the service command packet.
Step S132, when the complete response packet is not obtained, obtaining a response timeout time corresponding to the service command packet.
Step S133, judging whether the response timeout time is greater than a preset timeout duration.
Step S134, returning to the step of: judging whether a complete response packet corresponding to the service command packet can be obtained locally according to the identity of the service command packet until the response timeout time is longer than the preset timeout time or until the complete response packet corresponding to the service command packet is obtained locally according to the identity of the service command packet.
In this embodiment, the response timeout time refers to a time when each service command packet starts to be calculated from when it is sent from the data processing system. For example, the service command packet corresponding to the service identifier A1 is sent out at the time of 12 hours 58 minutes 00 seconds, and the current time is 12 hours 58 minutes 01 seconds, so that the response timeout time of the service command packet at the current time is 1 second.
The preset timeout period refers to a period of time preset to allow the service command packet to return in consideration of normal delay, that is, the service command packet can be normally received within the preset timeout period. The preset timeout period may be set according to the service type, or may be set according to the current network condition.
After the data processing system obtains the complete response packet corresponding to the service command packet, the data processing system temporarily stores the complete response packet in the local area, so that after the data processing system sends each service command packet to the server, the data processing system can search whether the corresponding complete response packet is temporarily stored locally according to the identity of each service command packet, if the complete response packet corresponding to a certain service command packet is obtained, the data processing system can perform relevant service processing, if the complete response packet corresponding to a certain service command packet is not obtained, the data processing system firstly obtains the response timeout time corresponding to the service command packet, judges whether the response timeout time corresponding to the service command packet is smaller than the preset timeout time, and if the response timeout time is smaller than the preset timeout time, repeatedly judges whether the complete response packet corresponding to the service command packet can be obtained locally until the response timeout time is larger than the preset timeout time or the complete response packet corresponding to the service command packet is obtained locally. If the response timeout time is longer than the preset timeout time, the complete response packet corresponding to the service command packet is still not acquired, the data processing system can generate an exception prompt to report that the service command packet is abnormal, and the user can conveniently conduct exception processing.
Referring to fig. 3, fig. 3 is a block diagram illustrating a structure of each part of a data processing system according to an embodiment of the present invention, and as shown in fig. 3, the data processing system may specifically include a user interaction module, a service command packet encapsulation module, a service command packet sending module, a service result information receiving module, and a data storage module.
With reference now to FIG. 4, FIG. 4 is a functional block diagram of portions of a data processing system according to one embodiment of the present invention, as depicted in FIG. 4, the functions of the portions are specifically as follows:
1. a user initiates a service at a user interaction module, and the user interaction module transmits service parameters to a service command packet packaging module;
2. the service command packet encapsulation module encapsulates the service parameters, sets a preset timeout duration, an identity identifier and a user identifier, generates a service command packet, sends the service command packet to a packet sending queue of the service command packet sending module, and simultaneously returns the identity identifier and the user identifier to the user interaction module;
3. the service command packet sending module obtains the service command packet in the packet sending queue, sends the service command packet to the server, starts to calculate response timeout time at the same time, and stores the response timeout time in a key value pair mode (the identity and the user are used as keys, and the response timeout time is used as a value);
4. The service result information receiving module starts a thread to receive original response packets corresponding to service command packets returned by the server in real time, and analyzes each original response packet to obtain an identity, the total number of packet fragments and the current fragment number;
5. the service result information receiving module judges whether the complete response packet corresponding to each service command packet is fragmented, if not, the original response packet corresponding to the service command packet is directly used as the complete response packet, the complete response packet is stored in the data storage module in a key value pair mode (the identity mark and the user mark are keys, the complete data packet is a value), and if fragmented, the original response packet corresponding to the same identity mark is stored in a key value pair mode (the identity mark and the user mark are keys, the set is a value, and the original response packet is stored in the set);
6. the business result information receiving module judges whether the total number of the original response packets corresponding to the same identity mark stored in the key value pair is equal to the total number of the packet fragments corresponding to the identity mark, if so, all the original response packets stored in the key value pair are sequenced and combined to obtain a complete response packet, the complete response packet is stored in the data storage module in a key value pair mode (the identity mark and the user mark are keys, and the complete data packet is a value), and if not, the step 4 is returned;
7. After receiving the identity and the user identifier returned by the service command packet packaging module, the user interaction module acquires a complete response packet corresponding to the identity and the user identifier from the data storage module according to the identity and the user identifier, if the complete response packet is not acquired, acquires response timeout time from the service command packet sending module according to the identity and the user identifier, judges whether the response timeout time is greater than a preset timeout time, if so, the data processing system performs exception processing, and if so, the user interaction module acquires the complete response packet corresponding to the identity and the user identifier from the data storage module again according to the identity and the user identifier until the response timeout time is greater than the preset timeout time. During this time, if a complete response packet is acquired, the service is completed according to the related data processing.
It should be noted that, for simplicity of description, the method embodiments are shown as a series of acts, but it should be understood by those skilled in the art that the embodiments are not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the embodiments. Further, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred embodiments, and that the acts are not necessarily required by the embodiments of the invention.
Referring to fig. 5, fig. 5 shows a data processing apparatus 50 according to an embodiment of the present invention, based on the same technical concept, applied to a data processing system, the data processing system being communicatively connected to a server, the apparatus comprising:
a first receiving module 51, configured to receive at least one service parameter input by a user;
a first determining module 52, configured to determine, according to at least one service parameter, a service command packet corresponding to each service parameter, where one service command packet carries a unique identity;
a sending module 53, configured to send each service command packet to the server;
the second receiving module 54 is configured to receive an original response packet corresponding to each service command packet returned by the server, where the original response packet and the service command packet that correspond to each other carry the same identity;
and the second determining module 55 is configured to determine complete response packets corresponding to the service command packets respectively according to the identity identifier, the total number of packet fragments and the current fragment number carried by the original response packets.
Optionally, the service parameter includes a service identifier and service data, and the first determining module includes:
The first generation sub-module is used for assigning a value to each service identifier to generate an identity identifier corresponding to each service identifier, and each identity identifier has uniqueness;
and the second generation sub-module is used for packaging the service data according to the identity mark and a preset packaging rule to generate a service command packet corresponding to the service parameter.
Optionally, the apparatus further comprises:
the first judging module is used for judging whether the complete response packet corresponding to each service command packet is fragmented or not according to the total number of packet fragments carried by each original response packet;
the second determining module includes:
the first obtaining submodule is used for taking an original response packet corresponding to the service command packet as a complete response packet corresponding to the service command packet under the condition that the complete response packet corresponding to the service command packet is processed in a slicing way;
and the second obtaining submodule is used for sequencing and combining the original response packets according to the current fragment numbers of the original response packets and the current fragment numbers of the original response packets according to all the original response packets corresponding to the same identity under the condition that the data packets corresponding to the service command packets are not fragmented, so as to obtain complete response packets corresponding to the service command packets.
Optionally, the apparatus further comprises:
the creation module is used for creating the association relation between the identity of the service command packet and the response packet storage queue under the condition that the complete response packet corresponding to the service command packet is processed in a fragmented manner;
the storage module is used for storing all original response packets corresponding to the identity mark into the response packet storage queue;
the second judging module is used for judging whether the number of the original response packets in the response packet storage queue is equal to the total number of the packet fragments;
and the third determining module is used for taking the original response packets in the response packet storage queue as all original response packets corresponding to the same identity mark under the condition that the number of the original response packets in the response packet storage queue is equal to the total number of the packet fragments.
Optionally, the apparatus further comprises:
the third judging module is used for judging whether the complete response packet corresponding to the service command packet can be locally acquired according to the identity of the service command packet;
the obtaining module is used for obtaining response timeout time corresponding to the service command packet under the condition that the complete response packet is not obtained;
a fourth judging module, configured to judge whether the response timeout time is greater than a preset timeout duration;
The returning module is configured to return to the step if the response timeout time is less than the preset timeout duration: judging whether a complete response packet corresponding to the service command packet can be obtained locally according to the identity of the service command packet until the response timeout time is longer than the preset timeout time or until the complete response packet corresponding to the service command packet is obtained locally according to the identity of the service command packet. .
The embodiment of the invention also provides a data processing device, which comprises:
one or more processors; and
one or more computer-readable storage media having instructions stored thereon that, when executed by the one or more processors, cause the apparatus to perform a data processing method according to any of the embodiments of the present invention.
The embodiment of the invention also provides a computer readable storage medium, and a computer program stored in the storage medium causes a processor to execute the data processing method according to the embodiment of the invention.
For the data processing apparatus embodiment, since it is substantially similar to the data processing method embodiment, the description is relatively simple, and the relevant points will be referred to in the description of the data processing method embodiment.
In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described by differences from other embodiments, and identical and similar parts between the embodiments are all enough to be referred to each other.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiment and all such alterations and modifications as fall within the scope of the embodiments of the invention.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or terminal. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or terminal device comprising the element.
The foregoing has outlined a detailed description of a data processing method, a data processing apparatus, and a computer readable storage medium, wherein specific examples are provided herein to illustrate the principles and embodiments of the present invention and to help understand the method and core idea of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
Claims (10)
1. A data processing method, applied to a data processing system, the data processing system and a server being connected through TCP socket communication, the method comprising:
receiving at least one business parameter input by a user;
determining service command packets corresponding to the service parameters according to at least one service parameter, wherein one service command packet carries a unique identity;
sending each service command packet to the server;
receiving original response packets corresponding to the service command packets respectively returned by the server, wherein the corresponding original response packets and the service command packets carry the same identity;
determining complete response packets corresponding to the service command packets respectively according to the identity marks carried by the original response packets, the total number of the packet fragments, the current fragment number and the user marks; and the user identifier is obtained by the data processing system from the connection information of the TCP socket connection corresponding to the original response packet.
2. The method of claim 1, wherein the service parameters include a service identifier and service data, and determining a service command packet corresponding to each service parameter according to at least one service parameter comprises:
Assigning a value to each service identifier to generate an identity identifier corresponding to each service identifier, wherein each identity identifier has uniqueness;
and packaging the service data according to the identity and a preset packaging rule to generate a service command packet corresponding to each service parameter.
3. The method of claim 1, wherein before determining complete response packets corresponding to each service command packet according to the identity, the total number of packet fragments, and the current fragment number carried by each original response packet, the method further comprises:
judging whether the complete response packet corresponding to each service command packet is fragmented or not according to the total number of packet fragments carried by each original response packet;
the determining the complete response packet corresponding to each service command packet according to the identity, the total number of packet fragments and the current fragment number carried by each original response packet includes:
under the condition that the complete response packet corresponding to the service command packet is processed in a slicing way, taking the original response packet corresponding to the service command packet as the complete response packet corresponding to the service command packet;
and under the condition that the data packet corresponding to the service command packet is not fragmented, sequencing and merging the original response packets according to the current fragment numbers of the original response packets and according to all the original response packets corresponding to the same identity mark to obtain a complete response packet corresponding to the service command packet.
4. The method of claim 3, wherein before sorting and merging each original response packet according to the current fragment number of each original response packet according to all original response packets corresponding to the same identity, and obtaining a complete response packet corresponding to the service command packet, the method further comprises:
under the condition that the complete response packet corresponding to the service command packet is processed in a fragmented way, establishing an association relation between the identity of the service command packet and a response packet storage queue;
storing all original response packets corresponding to the identity mark into the response packet storage queue;
judging whether the number of original response packets in the response packet storage queue is equal to the total number of packet fragments;
and under the condition that the number of the original response packets in the response packet storage queue is equal to the total number of the packet fragments, taking the original response packets in the response packet storage queue as all the original response packets corresponding to the same identity.
5. The method of claim 1, wherein after each of the service command packets is sent to the server, the method further comprises:
judging whether a complete response packet corresponding to the service command packet can be locally acquired according to the identity of the service command packet;
Acquiring response timeout time corresponding to the service command packet under the condition that the complete response packet is not acquired;
judging whether the response timeout time is greater than a preset timeout time;
and returning to the step under the condition that the response timeout time is smaller than the preset timeout time: judging whether a complete response packet corresponding to the service command packet can be obtained locally according to the identity of the service command packet until the response timeout time is longer than the preset timeout time or until the complete response packet corresponding to the service command packet is obtained locally according to the identity of the service command packet.
6. A data processing apparatus for use with a data processing system, the data processing system being communicatively coupled to a server via a TCP socket, the apparatus comprising:
the first receiving module is used for receiving at least one business parameter input by a user;
the first determining module is used for determining service command packets corresponding to the service parameters according to at least one service parameter, wherein one service command packet carries a unique identity;
the sending module is used for sending each service command packet to the server;
The second receiving module is used for receiving original response packets corresponding to the service command packets respectively returned by the server, wherein the original response packets and the service command packets which correspond to each other carry the same identity;
the second determining module is used for determining complete response packets corresponding to the service command packets respectively according to the identity marks carried by the original response packets, the total number of the packet fragments, the current fragment number and the user mark; and the user identifier is obtained by the data processing system from the connection information of the TCP socket connection corresponding to the original response packet.
7. The apparatus of claim 6, wherein the service parameters comprise a service identification and service data, and wherein the first determining module comprises:
the first generation sub-module is used for assigning a value to each service identifier to generate an identity identifier corresponding to each service identifier, and each identity identifier has uniqueness;
and the second generation sub-module is used for packaging the service data according to the identity mark and a preset packaging rule to generate a service command packet corresponding to the service parameter.
8. The apparatus of claim 6, wherein the apparatus further comprises:
The first judging module is used for judging whether the complete response packet corresponding to each service command packet is fragmented or not according to the total number of packet fragments carried by each original response packet;
the second determining module includes:
the first obtaining submodule is used for taking an original response packet corresponding to the service command packet as a complete response packet corresponding to the service command packet under the condition that the complete response packet corresponding to the service command packet is processed in a slicing way;
and the second obtaining submodule is used for sequencing and combining the original response packets according to the current fragment numbers of the original response packets and the current fragment numbers of the original response packets according to all the original response packets corresponding to the same identity under the condition that the data packets corresponding to the service command packets are not fragmented, so as to obtain complete response packets corresponding to the service command packets.
9. A data processing apparatus, comprising:
one or more processors; and
one or more computer-readable storage media having instructions stored thereon that, when executed by the one or more processors, cause the apparatus to perform the data processing method of any of claims 1 to 5.
10. A computer-readable storage medium, characterized in that it stores a computer program causing a processor to execute the data processing method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010270735.1A CN111597057B (en) | 2020-04-08 | 2020-04-08 | Data processing method, device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010270735.1A CN111597057B (en) | 2020-04-08 | 2020-04-08 | Data processing method, device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111597057A CN111597057A (en) | 2020-08-28 |
| CN111597057B true CN111597057B (en) | 2023-05-02 |
Family
ID=72187405
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010270735.1A Active CN111597057B (en) | 2020-04-08 | 2020-04-08 | Data processing method, device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111597057B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112395011B (en) * | 2020-11-24 | 2022-11-29 | 海宁奕斯伟集成电路设计有限公司 | Method for returning command response information, return control device and electronic equipment |
| CN112988812B (en) * | 2021-03-10 | 2024-02-06 | 京东科技控股股份有限公司 | Inventory data processing method, device, equipment and storage medium |
| CN113422754A (en) * | 2021-04-01 | 2021-09-21 | 阿里巴巴新加坡控股有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN114401298B (en) * | 2021-12-16 | 2023-03-21 | 荣耀终端有限公司 | Data transmission method, router, device, system and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017041398A1 (en) * | 2015-09-08 | 2017-03-16 | 安一恒通（北京）科技有限公司 | Data transmission method and device |
| CN106936867A (en) * | 2015-12-29 | 2017-07-07 | 阿里巴巴集团控股有限公司 | The response method and device of a kind of service request |
| CN106973074A (en) * | 2016-01-13 | 2017-07-21 | 腾讯科技（深圳）有限公司 | A kind of data processing method, apparatus and system |
| CN107342983A (en) * | 2017-06-09 | 2017-11-10 | 深圳震有科技股份有限公司 | A kind of transactional handles the method and system of the efficient UDP communications of more subpackages |
| CN107463576A (en) * | 2016-06-03 | 2017-12-12 | 上海点融信息科技有限责任公司 | Data processing method and equipment |
| CN110381030A (en) * | 2019-06-20 | 2019-10-25 | 视联动力信息技术股份有限公司 | A kind of processing method and processing device of synchronization request |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7908346B2 (en) * | 2009-04-20 | 2011-03-15 | International Business Machines Corporation | Processing a plurality of requests simultaneously in a web application |
-
2020
- 2020-04-08 CN CN202010270735.1A patent/CN111597057B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017041398A1 (en) * | 2015-09-08 | 2017-03-16 | 安一恒通（北京）科技有限公司 | Data transmission method and device |
| CN106936867A (en) * | 2015-12-29 | 2017-07-07 | 阿里巴巴集团控股有限公司 | The response method and device of a kind of service request |
| CN106973074A (en) * | 2016-01-13 | 2017-07-21 | 腾讯科技（深圳）有限公司 | A kind of data processing method, apparatus and system |
| CN107463576A (en) * | 2016-06-03 | 2017-12-12 | 上海点融信息科技有限责任公司 | Data processing method and equipment |
| CN107342983A (en) * | 2017-06-09 | 2017-11-10 | 深圳震有科技股份有限公司 | A kind of transactional handles the method and system of the efficient UDP communications of more subpackages |
| CN110381030A (en) * | 2019-06-20 | 2019-10-25 | 视联动力信息技术股份有限公司 | A kind of processing method and processing device of synchronization request |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111597057A (en) | 2020-08-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111597057B (en) | Data processing method, device and computer readable storage medium | |
| CN111245916B (en) | Service calling method, service arranging method and device based on micro-service gateway | |
| CN106201794B (en) | A kind of detection method and device of server | |
| CN109495306B (en) | Method and equipment for expanding capacity of service network | |
| EP3249882A1 (en) | Data uploading method, device and system | |
| CN109815256A (en) | A kind of data processing method, device, electronic equipment and storage medium | |
| CN110289983B (en) | Load balancing application creation method and device, computer equipment and storage medium | |
| US20200327045A1 (en) | Test System and Test Method | |
| CN111711544B (en) | Link dial testing method and device, electronic equipment and storage medium | |
| KR20100101170A (en) | Method and system for content categorization | |
| CA3129986A1 (en) | Network testing method and device | |
| CN115168031A (en) | Fog calculation system, method, electronic equipment and storage medium | |
| CN108966041B (en) | Audio and video playing point burying method, device, equipment and computer readable storage medium | |
| US20210067824A1 (en) | Video stream check method and system, computer device and storage medium | |
| CN111698281B (en) | Resource downloading method and device, electronic equipment and storage medium | |
| CN115396501B (en) | Information processing method and device, electronic equipment and readable storage medium | |
| CN108234658B (en) | Method and device for sensing health condition of server cluster and server | |
| CN116361155A (en) | Method and device for testing software development kit, storage medium and electronic device | |
| CN115118955A (en) | Service testing method, device and storage medium | |
| CN114443334A (en) | Method and device for reproducing fault information, equipment for reproducing fault and storage medium | |
| CN115391127A (en) | Dial testing method and device, storage medium and chip | |
| CN113595834B (en) | Method and device for detecting data processing capacity of video networking equipment | |
| CN115134618B (en) | Live stream life cycle information processing method and device and computing equipment | |
| CN114222002B (en) | Service request processing method and device, storage medium and electronic equipment | |
| CN112732757B (en) | Method, system, device, equipment and storage medium for processing degraded data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |