CN110798366A - Task logic processing method, device and equipment - Google Patents

Task logic processing method, device and equipment Download PDF

Info

Publication number
CN110798366A
CN110798366A CN201810862124.9A CN201810862124A CN110798366A CN 110798366 A CN110798366 A CN 110798366A CN 201810862124 A CN201810862124 A CN 201810862124A CN 110798366 A CN110798366 A CN 110798366A
Authority
CN
China
Prior art keywords
task
processing
socket
event
data packet
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810862124.9A
Other languages
Chinese (zh)
Other versions
CN110798366B (en
Inventor
田世坤
朱凌俊
石博
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Cloud Computing Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201810862124.9A priority Critical patent/CN110798366B/en
Publication of CN110798366A publication Critical patent/CN110798366A/en
Application granted granted Critical
Publication of CN110798366B publication Critical patent/CN110798366B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00Arrangements for monitoring or testing data switching networks
    • H04L43/10Active monitoring, e.g. heartbeat, ping or trace-route

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Cardiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

The application discloses a task logic processing method, which comprises the following steps: determining task logic contained in the task; the task logic contained by the task is run within one thread. By adopting the method, the problems of increasing thread switching overhead and processing time delay in the prior art are solved.

Description

Task logic processing method, device and equipment
Technical Field
The application relates to the field of network communication, in particular to a task logic processing method and device, electronic equipment and storage equipment.
Background
In a cloud computing environment, computing resources of a data center are divided into a large number of Virtual Machines (VMs) through a virtualization technology, a user flexibly deploys own applications in the VMs, such as web, social, game, finance and the like, some of the applications store important data, the data read-write performance is required to be good, the running is stable, 7 < 24-hour uninterrupted service can be achieved, some applications require good enough data reliability, the data have multiple redundant backups, the use is not affected even if a single server is down, some applications require high performance, the single-disk IOPS (number of read-write requests processed per second) is high, and Latency is low.
For a scene with high service availability requirement, accessing a VM disk to distributed storage with a redundancy characteristic is a good choice, and after a failure occurs in components such as a disk and a network, the distributed storage still can provide good availability. The data center is internally composed of a large number of clusters, in order to improve the selling rate, reduce resource contention and the like, the clusters are divided into a computing cluster and a storage cluster, a virtualization platform is deployed on each server (computing node) of the computing cluster, the user VM runs on the computing nodes, and the storage cluster deploys distributed storage and serves as a back-end storage to provide data storage service for the VM on the computing nodes. The IO (read-write request) processing of the user VM needs to go through several network communications, so the performance of the network communications plays a crucial role in the IO performance of the user VM.
With the development of network hardware, 25/40/100Gbps high-performance network cards are gradually applied to data center networks, and in order to implement high performance in networks between computing nodes and storage nodes, these high-performance network devices are also deployed, and to fully exert the performance advantages of these designs, software needs to be implemented by a polling-mode user-mode network protocol stack instead of the traditional interrupt-mode kernel-mode implementation, and the polling-mode user-mode network protocol stack implementation has smaller resource occupation, lower processing delay, and is easier to maintain. A user mode network protocol stack in a polling mode processes business logic related to VM disk IO, package of network RPC requests, data packet processing of a protocol stack (such as TCP/IP), processing of network card receiving and sending packets (such as DPDK) and the like in a plurality of threads of a process in a CPU polling mode, and the complex logics need to process factors such as CPU cache (CPU cache), network card receiving and sending efficiency and the like to improve the efficiency, wherein if the polling interval is shorter, the network performance is higher.
In the prior art, service logic processing, network RPC request packaging, protocol stack packet processing and network card receiving and sending packets are completed in a plurality of thread polling, which causes the problems of thread switching overhead and processing time delay increase.
Disclosure of Invention
The application provides a task logic processing method and device, an electronic device and a storage device, and aims to solve the problems that in the prior art, the thread switching overhead is increased and the processing time delay is increased.
The application provides a task logic processing method, which comprises the following steps:
determining task logic contained in the task;
the task logic contained by the task is run within one thread.
Optionally, the method further includes: all task logic contained in the task is processed in an event form.
Optionally, the task includes at least one of the following task logics:
event processing;
a protocol stack receives and transmits packets;
executing task logic contained in the task within a thread, comprising: event processing and protocol stack forwarding and receiving packets are run in one thread.
Optionally, the protocol stack transceiving packets include:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack;
processing a timing task of a protocol stack by using a thread timer;
performing packet sending processing by using a protocol stack;
and sending the encapsulated network data packet to the network card through the DPDK.
Optionally, the event processing includes:
processing a connection event; the connection event is an event triggered after a network data packet is analyzed by using a protocol stack;
RPC communication processing;
and (5) service processing.
Optionally, the connection event includes at least one of the following events:
socket readable events, socket writable events, socket setup events, socket disconnect events.
Optionally, the running of the task logic in the thread includes executing at least one task contained in the following tasks in one thread:
sending a task by a client; the server receives the task; the server sends a task; and the client receives the task.
Optionally, the client sends the task, including the following steps:
packaging the request context into an RPC message object;
performing RPC communication processing on the RPC message object, and serializing the RPC message object into memory data;
writing the memory data to a TCP/IP socket interface through a socket writable event;
the memory data is taken out from a TCP/IP socket interface by utilizing a protocol stack and is packaged into a network data packet;
and submitting the network data packet to the network card through the DPDK and sending the network data packet to the server.
Optionally, the step of receiving the task by the server includes:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack to obtain payload data and generate a socket readable event;
reading payload data through the socket-readable event;
decoding the read payload data to obtain a corresponding RPC message object;
obtaining a request context from the RPC message object;
optionally, the server sends a task, including the following steps:
performing business logic processing after obtaining a request result, and packaging the request result into an RPC message object;
performing RPC communication processing on the RPC message object, and serializing the RPC message object into memory data;
writing the memory data to a TCP/IP socket interface through a socket writable event;
the memory data is taken out from a TCP/IP socket interface by utilizing a protocol stack and is packaged into a network data packet;
and submitting the network data packet to a network card through the DPDK and sending the network data packet to a client.
Optionally, the receiving of the task by the client includes the following steps:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack to generate a socket readable event;
reading payload data through the socket-readable event;
decoding the read payload data to obtain a corresponding RPC message object;
and obtaining a request result from the RPC message object.
The present application further provides a processing apparatus for task logic, including:
the task logic determination unit is used for determining task logic contained in the task;
and the task logic running unit is used for running the task logic contained in the task in one thread.
The present application further provides an electronic device, comprising:
a processor; and
a memory for a program of a processing method of a task logic, the device performing the following steps after being powered on and running the program of the processing method of the task logic by the processor:
determining task logic contained in the task;
the task logic contained by the task is run within one thread.
The present application also provides a storage device storing a program of a processing method of task logic, the program being executed by a processor and performing the steps of:
determining task logic contained in the task;
the task logic contained by the task is run within one thread.
Compared with the prior art, the method has the following advantages:
the application provides a processing method and device of task logic, electronic equipment and storage equipment, the task logic contained in the task is operated in one thread, the switching overhead between the threads is reduced, and the processing time delay is reduced.
Drawings
Fig. 1 is a flowchart of a processing method of task logic according to a first embodiment of the present application.
Fig. 2 is a flowchart of logic for running all tasks in one thread according to a first embodiment of the present application.
Fig. 3 is a schematic diagram of a processing device of task logic according to a second embodiment of the present application.
Fig. 4 is a schematic diagram of an electronic device according to a third embodiment of the present application.
Detailed Description
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit of this application and is therefore not limited to the specific implementations disclosed below.
The first embodiment of the application provides a task logic processing method. The following is described in detail with reference to fig. 1 and 2.
As shown in fig. 1, in step S101, task logic included in the task is determined.
The task comprises task logic including: event processing; the protocol stack sends and receives packets.
Event processing: after the RPC layer is packaged, the RPC request and response are processed, the data is transmitted and received on a network connection correspondingly, in order to realize high performance and large scale of network performance, the data is transmitted and received by realizing event-driven asynchronization, and the event processing mainly processes various network events, such as a socket establishing event, a socket disconnecting event, a socket readable event, a socket writable event and the like.
The RPC layer realizes the processing of various network requests of services, the network requests of the services comprise four elements of method, request, response and result, before sending, a described service scene needs to be packaged into a request message object (mainly a google protobuf object), and when sending, the object is serialized into a memory and sent to a remote end through TCP/IP; after receiving the bare data obtained by the TCP/IP, the remtote deserializes the bare data into a request object, then the service logic is conveniently processed, and the response is encapsulated and sent to the requesting party.
In short, RPC can define different types of method/request/response for different services according to their own needs, and RPC layer encapsulation mainly refers to serialization and deserialization of request/response, such as IO service, data processing service, proxy service, and the like.
Wherein, the event processing comprises:
processing a connection event; RPC communication processing; IO traffic handling, etc.
The connection event in the connection event processing is an event triggered after a network data packet is analyzed by using a protocol stack, and the connection event comprises the following events: a socket readable event, a socket writable event, a socket established event, a socket disconnected event, etc.
The IO service processing comprises the following steps: a server end (server end) submits an IO request to a bottom layer for storage, a client end (client end) notifies an upper layer service IO request processing result, and the like. Specifically, the IO service processing includes the following operations: opening and closing a disk, analyzing an IO request, controlling IO flow, processing stripes, performing crc (Cyclic Redundancy Check), monitoring performance and the like.
RPC (Remote Procedure Call) communication processing: after the IO service processing is completed, based on the open source google protocol packet as an RPC request, RPC communication encapsulates the service request into different services and methods, for example, a class of services is a class of services when a disk is opened or closed; the IO request is a type of service; the IO requests are divided into different methods such as read and write, and are packaged as RPC request and response.
Wherein, the protocol stack receives and dispatches the package, includes: receiving a network data packet from a network card through a DPDK; analyzing and processing the network data packet by using a protocol stack; processing a timing task by using a protocol stack; utilizing a protocol stack to perform packet sending processing on the encapsulated network data packet; and sending the packaged network data packet to a network card and the like through the DPDK.
Receiving a network data packet from the network card via the DPDK: receiving network data packets from the network card via the DPDK, and trying to receive a batch (a plurality of) of network data packets from the network card every time;
analyzing and processing the network data packet by using a protocol stack: analyzing the headers of a link layer, an IP layer and a TCP/UDP layer in sequence for the received network data packet; processing life cycles of connection establishment, destruction and the like; recording payload data (raw data), processing ack (acknowledge character), fast retransmission, etc. on the socket connection; after the processing, corresponding connection events are generated, such as socket readable events, socket writable events, socket establishment events, socket disconnection events, and the like.
Processing timing tasks of a protocol stack by using a thread timer: timeout retransmission, connection timewait, etc. Because the TCP protocol stack needs to ensure the reliable transmission of data, the scenes of packet loss and disorder need to be retried regularly; the TCP connection has a state, and if the state is abnormal, a timing cleaning is required, such as timewait.
And (3) utilizing a protocol stack to perform packet sending processing: processing socket call events, such as socket setup events, socket disconnect events, write data events, and the like; and encapsulating the data packets into network data packets, and encapsulating the headers of a TCP/UDP layer, an IP layer and a link layer in sequence.
Sending the encapsulated network data packet to the network card through the DPDK: and sending the packaged network data packets to the network card through the DPDK, and trying to send a batch of network data packets to the network card every time.
As shown in fig. 1, in step S102, task logic included in a task is executed in one thread.
The task logic contained in the task is operated in one thread, so that thread notification and CPU context switching back and forth for multiple times are reduced, the CPU cache miss (cache miss) is reduced, the function of the CPU cache is fully exerted, and the execution efficiency of the CPU instruction is improved; and further reducing the memory copy in the protocol stack receiving and sending packet logic, reducing the operation task of the CPU, reducing the switching overhead among CPU load threads and reducing the processing delay.
The task logic for running the task in one thread comprises the following steps: event processing and protocol stack forwarding and receiving packets are run in one thread.
Preferably, in order to implement the task logic contained in the task running in one thread and ensure that the task logic contained in the task does not block, all the task logic (such as service processing, RPC communication, protocol stack send-receive packets, and the like) contained in the task running in one thread is processed in an event form.
The task logic for running the task in one thread comprises the task logic for executing at least one of the following tasks in one thread:
sending a task by a client; the server receives the task; the server sends a task; and the client receives the task.
The client sends the task, and the method can comprise the following steps:
packaging the request context into an RPC message object;
performing RPC communication processing on the RPC message object, and serializing the RPC message object into memory data;
writing the memory data to a TCP/IP socket interface through a socket writable event;
the memory data is taken out from a TCP/IP socket interface by utilizing a protocol stack and is packaged into a network data packet;
and submitting the network data packet to the network card through the DPDK and sending the network data packet to the server.
The task receiving by the server side can comprise the following steps:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack to obtain payload data and generate a socket readable event;
reading payload data through the socket-readable event;
decoding the read payload data to obtain a corresponding RPC message object;
obtaining a request context from the RPC message object;
the server sends the task, which may include the following steps:
after the request result is obtained, business logic processing is carried out, and the request result is packaged into a second RPC message object;
performing RPC communication processing on the RPC message object, and serializing the RPC message object into memory data;
writing the memory data to a TCP/IP socket interface through a socket writable event;
the memory data is taken out from a TCP/IP socket interface by utilizing a protocol stack and is packaged into a network data packet;
and submitting the network data packet to a network card through the DPDK and sending the network data packet to a client.
The client receives tasks, and the method can comprise the following steps:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack to generate a socket readable event;
reading payload data through the socket-readable event;
decoding the read payload data to obtain a corresponding RPC message object;
and obtaining a request result from the RPC message object.
The client sending task is introduced by taking VM storage IO service as an example; the server receives the task; the server sends a task; and the client receives the task. Referring to fig. 2, the process of sending tasks by the client is as follows: the flow represented by the dotted line is processed first: starting event processing, processing IO service, processing RPC communication, processing connection event, and ending event processing; and after the event processing is finished, the polling is started, the protocol stack sends a packet, the packet is sent to the network card through the DPDK, and the polling is finished. The process of sending the tasks by the server is similar to that of sending the tasks by the client; the process of receiving the task by the server is opposite to the process of sending the task by the client, and the details are described below.
1. Client-side task (client-side submitting I/O request)
1) And IO service processing: after obtaining the event of the Request arrival from the Request queue, the client takes out the Request context (such as seven basic attributes of IO service stored by a VM), and then encapsulates the Request context into an RPC message object, such as VMIO _ RPC _ Request; wherein the seven basic attributes include:
cluster _ id; /from which cluster
device _ id; /from which disk
request _ id; each request of the block of disks is given an id identification +
type; /read, write, sync, trim, etc. types
data; v. data Address
length; /' Io request length
offset; v. starting position of Io request
Besides the seven attributes, other attributes such as performance trace and data correctness-related attributes are also available and are not listed in detail here; the above seven are basic attributes of VM storage IO traffic.
2) RPC communication processing: adding an RPC header to an RPC message object (service and method are arranged inside) through the method of submitmIO; serializing the RPC message object into memory data (encoding the RPC message object into a section of memory buffer);
3) processing a connection event: taking out the data in the buffer through a socket writable event, and writing the data to a TCP/IP socket interface;
4) and (3) protocol stack packet sending processing: the TCP/IP protocol stack divides the data of the buffer into data with the size of a network MTU according to the requirement, and attaches a packet header of TCP/IP/ETHER;
5) sending a packet to the network card through the DPDK: and submitting the buffer with the TCP/IP/ETHER packet header to a network card through the DPDK, and sending the buffer to a server.
All the steps are executed in one thread.
2. Receiving task by server
Referring to fig. 2, the process flow is as follows:
1) the server receives a network data packet from the network card through the DPDK;
2) analyzing (analyzing the packet header of TCP/IP/ETHER) and processing the network data packet by using a protocol stack to obtain payload data and generate a socket readable event;
if necessary, processing protocol stack timing task processing, such as TCP timeout retransmission, connection timewait, etc. (the TCP protocol stack needs to ensure reliable transmission of data, and scenes such as packet loss and disorder need to be retried at regular time, and the TCP is connected with a state, and needs to be cleared at regular time when the state is abnormal, such as timewait);
3) reading payload data through the socket-readable event;
4) decoding the read payload data to obtain a corresponding RPC message object, such as VMIO _ RPC _ Request;
5) obtaining the seven basic attributes for obtaining the VM storage IO service from the VMIO _ RPC _ Request message object; and submitting the IO request to a corresponding distributed file.
3. The server sends a task:
1) after the corresponding distributed file returns a result, packaging the result into an RPC message object, such as VMIO _ RPC _ Response;
2) RPC communication adds RPC header to RPC message object; and serializing the RPC message object into memory data (encoding the RPC message object into a segment of memory buffer)
3) Taking out the data in the buffer through a socket writable event, and writing the data to a TCP/IP socket interface;
4) the TCP/IP protocol stack divides the data of the buffer into data with the size of a network MTU according to the requirement, and attaches a packet header of TCP/IP/ETHER;
5) and submitting the buffer with the TCP/IP/ETHER packet header to a network card through the DPDK, and sending the buffer to the client.
4. The client receives the tasks:
1) the client receives a network data packet from the network card through the DPDK;
2) analyzing (analyzing the packet header of TCP/IP/ETHER) and processing the network data packet by using a protocol stack to obtain payload data and generate a socket readable event;
if necessary, processing protocol stack timing task processing, such as TCP timeout retransmission, connection timewait, etc. (the TCP protocol stack needs to ensure reliable transmission of data, and scenes such as packet loss and disorder need to be retried at regular time, and the TCP is connected with a state, and needs to be cleared at regular time when the state is abnormal, such as timewait);
3) reading payload data through the socket-readable event;
4) decoding the read payload data to obtain a corresponding RPC message object, such as VMIO _ RPC _ Response;
5) the result of the VM storage IO Request is obtained from the VMIO _ RPC _ Request message object, for example, it may be determined whether the IO Request is successful or failed according to the error, and data indicating performance correlation is obtained from data such as trace.
It should be noted that, in one thread, the basic flow processing of each large flow (for example, the client receives the IO processing result, the client submits the IO request, and the like) is the same, and there are different business logic processing sub-flows according to the business needs.
One thread can simultaneously run a client and a server, the RPC communication processing flows of the client and the server are different, but the same client RPC processing flow and the same server RPC processing flow can be used for different services.
Because the first embodiment of the present application implements that one thread processes all logics of an IO lifecycle, a CPU core where the thread is located executes more instructions, which increases the load of the CPU, and the processing capability of one CPU core has an upper limit, if the load is too high and polling is not timely enough, the high-pressure scene network retransmission may increase, and the performance may deteriorate. Therefore, in order to solve the above problem, in the first embodiment of the present application, memory copy is respectively reduced once in the process of packet receiving processing and packet sending processing of the protocol stack:
the method comprises the steps of realizing send queue send _ queue and send queue maintenance send _ head, recording the current sending position (sequence is distributed when data is sent by Tcp, the current sending position refers to the sending sequence), directly creating DPDK buffer according to factors such as data size, MTU and sending window of user IO in the process of writing events, adding the DPDK buffer into a sending queue, and copying data into DPDKbuffer.
Realizing a receiving queue recv _ queue, maintaining data segment fragments formed by out-of-order receiving in the receiving queue, and maintaining a plurality of DPDKbuffers with continuous sequences in each fragment; in read event processing, data is copied directly from recv _ queue to the user's IO.
As a large amount of small-section data is processed and a header is attached during data receiving and sending of the TCP/IP network, and the send _ queue and the recv _ queue manage the data sections, data copying is reduced, and performance is optimized.
Corresponding to the above provided method for processing task logic, a second embodiment of the present application further provides a device for processing task logic.
As shown in fig. 3, the processing device of task logic includes: a task logic determination unit 301 and a task logic execution unit 302.
A task logic determination unit 301, configured to determine task logic included in a task;
a task logic execution unit 302, configured to execute task logic included in the task within one thread.
Optionally, the method further includes: all task logic contained in the task is processed in an event form.
Optionally, the task includes at least one of the following task logics:
event processing;
a protocol stack receives and transmits packets;
the running of the task logic contained in the task in one thread comprises: event processing and protocol stack forwarding and receiving packets are run in one thread.
Optionally, the protocol stack transceiving packets include:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack;
processing a timing task of a protocol stack by using a thread timer;
performing packet sending processing by using a protocol stack;
and sending the encapsulated network data packet to the network card through the DPDK.
Optionally, the event processing includes:
processing a connection event; the connection event is an event triggered after a network data packet is analyzed by using a protocol stack;
RPC communication processing;
and (5) service processing.
Optionally, the connection event includes at least one of the following events:
socket readable events, socket writable events, socket setup events, socket disconnect events.
Optionally, the task logic operation unit is specifically configured to:
executing within a thread task logic comprised of at least one of: :
sending a task by a client; the server receives the task; the server sends a task; and the client receives the task.
Optionally, the client sends the task, including the following steps:
packaging the request context into an RPC message object;
performing RPC communication processing on the RPC message object, and serializing the RPC message object into memory data;
writing the memory data to a TCP/IP socket interface through a socket writable event;
the memory data is taken out from a TCP/IP socket interface by utilizing a protocol stack and is packaged into a network data packet;
and submitting the network data packet to the network card through the DPDK and sending the network data packet to the server.
Optionally, the step of receiving the task by the server includes:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack to obtain payload data and generate a socket readable event;
reading payload data through the socket-readable event;
decoding the read payload data to obtain a corresponding RPC message object;
obtaining a request context from the RPC message object;
optionally, the server sends a task, including the following steps:
performing business logic processing after obtaining a request result, and packaging the request result into an RPC message object;
performing RPC communication processing on the RPC message object, and serializing the RPC message object into memory data;
writing the memory data to a TCP/IP socket interface through a socket writable event;
the memory data is taken out from a TCP/IP socket interface by utilizing a protocol stack and is packaged into a network data packet;
and submitting the network data packet to a network card through the DPDK and sending the network data packet to a client.
Optionally, the receiving of the task by the client includes the following steps:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack to generate a socket readable event;
reading payload data through the socket-readable event;
decoding the read payload data to obtain a corresponding RPC message object;
and obtaining a request result from the RPC message object.
It should be noted that, for the detailed description of the processing device of the task logic provided in the second embodiment of the present application, reference may be made to the related description of the first embodiment of the present application, and details are not repeated here.
Corresponding to the above-provided task logic processing method, a third embodiment of the present application further provides an electronic device.
As shown in fig. 4, the electronic apparatus includes:
a processor 401; and
a memory 402 for a program of a processing method of a task logic, the device being powered on and executing the program of the processing method of the task logic by the processor, the following steps being performed:
determining task logic contained in the task;
the task logic contained by the task is run within one thread.
Optionally, the method further includes: all task logic contained in the task is processed in an event form.
Optionally, the task includes at least one of the following task logics:
event processing;
a protocol stack receives and transmits packets;
the running of the task logic contained in the task in one thread comprises: event processing and protocol stack forwarding and receiving packets are run in one thread.
Optionally, the protocol stack transceiving packets include:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack;
processing a timing task of a protocol stack by using a thread timer;
performing packet sending processing by using a protocol stack;
and sending the encapsulated network data packet to the network card through the DPDK.
Optionally, the event processing includes:
processing a connection event; the connection event is an event triggered after a network data packet is analyzed by using a protocol stack;
RPC communication processing;
and (5) service processing.
Optionally, the connection event includes at least one of the following events:
socket readable events, socket writable events, socket setup events, socket disconnect events.
Optionally, the running task logic included in the task in one thread includes: executing within a thread task logic comprised of at least one of:
sending a task by a client; the server receives the task; the server sends a task; and the client receives the task.
Optionally, the client sends the task, including the following steps:
packaging the request context into an RPC message object;
performing RPC communication processing on the RPC message object, and serializing the RPC message object into memory data;
writing the memory data to a TCP/IP socket interface through a socket writable event;
the memory data is taken out from a TCP/IP socket interface by utilizing a protocol stack and is packaged into a network data packet;
and submitting the network data packet to the network card through the DPDK and sending the network data packet to the server.
Optionally, the step of receiving the task by the server includes:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack to obtain payload data and generate a socket readable event;
reading payload data through the socket-readable event;
decoding the read payload data to obtain a corresponding RPC message object;
obtaining a request context from the RPC message object;
optionally, the server sends a task, including the following steps:
performing business logic processing after obtaining a request result, and packaging the request result into an RPC message object;
performing RPC communication processing on the RPC message object, and serializing the RPC message object into memory data;
writing the memory data to a TCP/IP socket interface through a socket writable event;
the memory data is taken out from a TCP/IP socket interface by utilizing a protocol stack and is packaged into a network data packet;
and submitting the network data packet to a network card through the DPDK and sending the network data packet to a client.
Optionally, the receiving of the task by the client includes the following steps:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack to generate a socket readable event;
reading payload data through the socket-readable event;
decoding the read payload data to obtain a corresponding RPC message object;
and obtaining a request result from the RPC message object.
It should be noted that, for the detailed description of the electronic device provided in the third embodiment of the present application, reference may be made to the related description of the first embodiment of the present application, and details are not repeated here.
In correspondence with the above-described processing method for task logic, a fourth embodiment of the present application further provides a storage device storing a program of the processing method for task logic, where the program is executed by a processor to perform the following steps:
determining task logic contained in the task;
the task logic contained by the task is run within one thread.
It should be noted that, for the detailed description of the storage device provided in the fourth embodiment of the present application, reference may be made to the related description of the first embodiment of the present application, and details are not described here again.
Although the present application has been described with reference to the preferred embodiments, it is not intended to limit the present application, and those skilled in the art can make variations and modifications without departing from the spirit and scope of the present application, therefore, the scope of the present application should be determined by the claims that follow.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Claims (14)

1. A method for processing task logic is characterized by comprising the following steps:
determining task logic contained in the task;
the task logic contained by the task is run within one thread.
2. The method of claim 1, further comprising: the task comprises task logic which is processed in an event form.
3. The method of claim 1, wherein the task comprises at least one of the following task logic:
event processing;
a protocol stack receives and transmits packets;
the running of the task logic contained in the task in one thread comprises: event processing and protocol stack forwarding and receiving packets are run in one thread.
4. The method of claim 3, wherein the protocol stack is configured to send and receive packets, comprising:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack;
processing the timing task of the protocol stack by using a thread timer;
performing packet sending processing by using a protocol stack;
and sending the encapsulated network data packet to the network card through the DPDK.
5. The method of claim 3, wherein the event processing comprises:
processing a connection event; the connection event is an event triggered after a network data packet is analyzed by using a protocol stack;
RPC communication processing;
and (5) service processing.
6. The method of claim 5, wherein the connection event comprises at least one of:
socket readable events, socket writable events, socket setup events, socket disconnect events.
7. The method of any one of claims 1, 3, 4, and 5, wherein executing the task logic included in the task within a thread comprises executing the task logic included in at least one of the following tasks within a thread:
sending a task by a client; the server receives the task; the server sends a task; and the client receives the task.
8. The method of claim 7, wherein the client sends tasks, comprising the steps of:
packaging the request context into an RPC message object;
performing RPC communication processing on the RPC message object, and serializing the RPC message object into memory data;
writing the memory data to a TCP/IP socket interface through a socket writable event;
the memory data is taken out from a TCP/IP socket interface by utilizing a protocol stack and is packaged into a network data packet;
and submitting the network data packet to the network card through the DPDK and sending the network data packet to the server.
9. The method of claim 8, wherein the server receives the task, comprising the steps of:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack to obtain payload data and generate a socket readable event;
reading payload data through the socket-readable event;
decoding the read payload data to obtain a corresponding RPC message object;
obtaining a request context from the RPC message object.
10. The method of claim 8, wherein the server sends tasks, comprising the steps of:
performing business logic processing after obtaining a request result, and packaging the request result into an RPC message object;
performing RPC communication processing on the RPC message object, and serializing the RPC message object into memory data;
writing the memory data to a TCP/IP socket interface through a socket writable event;
the memory data is taken out from a TCP/IP socket interface by utilizing a protocol stack and is packaged into a network data packet;
and submitting the network data packet to a network card through the DPDK and sending the network data packet to a client.
11. The method of claim 8, wherein the client receives tasks, comprising the steps of:
receiving a network data packet from a network card through a DPDK;
analyzing and processing the network data packet by using a protocol stack to generate a socket readable event;
reading payload data through the socket-readable event;
decoding the read payload data to obtain a corresponding RPC message object;
and obtaining a request result from the RPC message object.
12. A processing apparatus of task logic, comprising:
the task logic determination unit is used for determining task logic contained in the task;
and the task logic running unit is used for running the task logic contained in the task in one thread.
13. An electronic device, comprising:
a processor; and
a memory for a program of a processing method of a task logic, the device performing the following steps after being powered on and running the program of the processing method of the task logic by the processor:
determining task logic contained in the task;
the task logic contained by the task is run within one thread.
14. A storage device, characterized in that,
a program storing a processing method of a task logic, the program being executed by a processor, and performing the steps of:
determining task logic contained in the task;
the task logic contained by the task is run within one thread.
CN201810862124.9A 2018-08-01 2018-08-01 Task logic processing method, device and equipment Active CN110798366B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810862124.9A CN110798366B (en) 2018-08-01 2018-08-01 Task logic processing method, device and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810862124.9A CN110798366B (en) 2018-08-01 2018-08-01 Task logic processing method, device and equipment

Publications (2)

Publication Number Publication Date
CN110798366A true CN110798366A (en) 2020-02-14
CN110798366B CN110798366B (en) 2023-02-24

Family

ID=69424990

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810862124.9A Active CN110798366B (en) 2018-08-01 2018-08-01 Task logic processing method, device and equipment

Country Status (1)

Country Link
CN (1) CN110798366B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113067849A (en) * 2021-02-05 2021-07-02 湖南国科亿存信息科技有限公司 Network communication optimization method and device based on Glusterfs
CN114978682A (en) * 2022-05-20 2022-08-30 北京紫光展锐通信技术有限公司 Data processing method, device and equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102542047A (en) * 2011-12-28 2012-07-04 奇智软件(北京)有限公司 Task logic library generating method, task logic library generating system, data processing method and data processing system
CN105426326A (en) * 2015-11-05 2016-03-23 上海斐讯数据通信技术有限公司 High-concurrency queue storage method and system
KR101702426B1 (en) * 2016-04-23 2017-02-03 광운대학교 산학협력단 Video transmission method based on multi HTTP threads for reducing the viewpoint change delay in multi-view video service
CN106980546A (en) * 2016-01-18 2017-07-25 阿里巴巴集团控股有限公司 A kind of task asynchronous execution method, apparatus and system
CN107181738A (en) * 2017-04-25 2017-09-19 中国科学院信息工程研究所 A kind of software implementation intruding detection system and method
CN108270676A (en) * 2016-12-31 2018-07-10 普天信息技术有限公司 A kind of network data processing method and device based on IntelDPDK

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102542047A (en) * 2011-12-28 2012-07-04 奇智软件(北京)有限公司 Task logic library generating method, task logic library generating system, data processing method and data processing system
CN105426326A (en) * 2015-11-05 2016-03-23 上海斐讯数据通信技术有限公司 High-concurrency queue storage method and system
CN106980546A (en) * 2016-01-18 2017-07-25 阿里巴巴集团控股有限公司 A kind of task asynchronous execution method, apparatus and system
KR101702426B1 (en) * 2016-04-23 2017-02-03 광운대학교 산학협력단 Video transmission method based on multi HTTP threads for reducing the viewpoint change delay in multi-view video service
CN108270676A (en) * 2016-12-31 2018-07-10 普天信息技术有限公司 A kind of network data processing method and device based on IntelDPDK
CN107181738A (en) * 2017-04-25 2017-09-19 中国科学院信息工程研究所 A kind of software implementation intruding detection system and method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113067849A (en) * 2021-02-05 2021-07-02 湖南国科亿存信息科技有限公司 Network communication optimization method and device based on Glusterfs
CN114978682A (en) * 2022-05-20 2022-08-30 北京紫光展锐通信技术有限公司 Data processing method, device and equipment

Also Published As

Publication number Publication date
CN110798366B (en) 2023-02-24

Similar Documents

Publication Publication Date Title
CN109725829B (en) System and method for end-to-end QoS solution for data storage system
EP1581875B1 (en) Using direct memory access for performing database operations between two or more machines
US10205801B2 (en) Data transfer policies between source and target servers in a wide area network
US11025564B2 (en) RDMA transport with hardware integration and out of order placement
WO2020052323A1 (en) Data storage method, device and server
US11068412B2 (en) RDMA transport with hardware integration
US11265235B2 (en) Technologies for capturing processing resource metrics as a function of time
CN114201421B (en) Data stream processing method, storage control node and readable storage medium
US10819641B2 (en) Highly available servers
US11379405B2 (en) Internet small computer interface systems extension for remote direct memory access (RDMA) for distributed hyper-converged storage systems
WO2022218160A1 (en) Data access system and method, and device and network card
WO2024037296A1 (en) Protocol family-based quic data transmission method and device
CN109857545B (en) Data transmission method and device
CN110798366B (en) Task logic processing method, device and equipment
CN115270033A (en) Data access system, method, equipment and network card
WO2024041576A1 (en) Live migration method and system for virtual machine, and device and storage medium
CN113411363A (en) Uploading method of image file, related equipment and computer storage medium
US10642667B1 (en) Apparatus, system, and method for efficiently sharing data between processes
US10673801B2 (en) Dynamic communication session management
US11568089B2 (en) Offloading operations from a primary processing device to a secondary processing device
CN111857964A (en) Method and equipment for transmitting data of virtual machine CAN port to physical machine CAN port equipment
US11212349B1 (en) Switching between network protocols for a data storage system
US11863318B2 (en) Error correction for network packets
CN116760850B (en) Data processing method, device, equipment, medium and system
US11442943B1 (en) Error-causing record isolation for data stream processing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 40023520

Country of ref document: HK

GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20230613

Address after: Room 1-2-A06, Yungu Park, No. 1008 Dengcai Street, Sandun Town, Xihu District, Hangzhou City, Zhejiang Province

Patentee after: Aliyun Computing Co.,Ltd.

Address before: Box 847, four, Grand Cayman capital, Cayman Islands, UK

Patentee before: ALIBABA GROUP HOLDING Ltd.