WO2017177400A1 - 一种数据处理方法及系统 - Google Patents

一种数据处理方法及系统 Download PDF

Info

Publication number
WO2017177400A1
WO2017177400A1 PCT/CN2016/079174 CN2016079174W WO2017177400A1 WO 2017177400 A1 WO2017177400 A1 WO 2017177400A1 CN 2016079174 W CN2016079174 W CN 2016079174W WO 2017177400 A1 WO2017177400 A1 WO 2017177400A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
processor
target data
network card
data buffer
Prior art date
Application number
PCT/CN2016/079174
Other languages
English (en)
French (fr)
Inventor
徐君
邵云峰
张丰伟
王元钢
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN201680007359.9A priority Critical patent/CN107615259B/zh
Priority to PCT/CN2016/079174 priority patent/WO2017177400A1/zh
Publication of WO2017177400A1 publication Critical patent/WO2017177400A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14—Handling requests for interconnection or transfer
    • G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
    • G06F13/28—Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal

Definitions

  • the present invention relates to the field of data processing systems, and in particular, to a data processing method and system.
  • Distributed storage system distributes data on multiple independent storage servers in a decentralized system. It uses multiple storage servers to share storage load. It not only improves system reliability, availability, and access efficiency. It is also easy to expand.
  • FIG. 1 is a schematic structural diagram of a storage node of a distributed storage system in the prior art.
  • the server of the storage node includes: a network card 101, a memory 102, a processor 103, an I/O bus 104, and a memory bus 105.
  • the network card 101, the memory 102, and the processor 103 are connected to the memory bus 105 through the I/O bus 104.
  • the network card 101 corresponds to the first data buffer area 1021
  • the processor 103 corresponds to the second data buffer area 1022.
  • the first data buffer area 1021 and the second data buffer area 1022 are both located in the memory 102, and the two are connected through the memory bus 105. .
  • the network card 101 when the client sends a data packet to the server, after receiving the target data, the network card 101 performs unpacking and network protocol stack processing on the target data, and then the network card 101 copies the processed target data to the first data buffer.
  • the interrupt is triggered at the same time, and the driver copies the target data in the first data buffer 1021 into the second data buffer 1022 by the interrupt for processing by the processor 103.
  • the embodiment of the invention provides a data processing method and system, which aims to reduce the number of times of copying target data to reduce the occupation of the memory bandwidth of the system.
  • a first aspect of an embodiment of the present invention provides a data processing method, the method being applied to a data processing system, the data processing system comprising a network card, a memory, and at least one processor, the network card being shared with the at least one processor a data buffer, the data buffer being located in the memory;
  • the method includes:
  • the network card receives target data
  • the network card writes the target data into the data buffer area
  • the at least one processor processes the target data of the data buffer.
  • the network card after receiving the target data, the network card performs unpacking and network protocol stack processing on the target data, and then the network card caches the processed target data in the network card and the processor shared data buffer area, and the processor processes the In the target data, it is not necessary to copy the target data from the data buffer to other data buffers, and only the target data is processed in the data buffer, thereby reducing the number of times of copying the target data, thereby reducing Occupancy of the system's memory bandwidth.
  • the memory is a storage level memory SCM.
  • the memory uses a non-volatile memory SCM, which enables the target data to be stored persistently.
  • the SCM can be used as the memory of the system or as the external storage of the system, which is not limited herein.
  • the data buffer area includes at least one sub-area, where the at least one a sub-region corresponding to the at least one processor;
  • the method further includes: the network card receiving an indication identifier, where the indication identifier is used to indicate a processor corresponding to the target data;
  • Writing, by the network card, the target data into the data buffer area includes:
  • Processing the target data of the data buffer by the at least one processor includes:
  • the processor indicated by the indication identifier processes the target data in a sub-region corresponding thereto.
  • the data buffer may be divided into a plurality of sub-areas, wherein the plurality of sub-areas are in one-to-one correspondence with the plurality of processors, and the network card further receives an indication identifier, where the indication identifier is used to indicate a processor corresponding to the target data, that is, When the client sends the target data to the server, it can specify which processor to process the target data or which sub-area the network card writes the target data to.
  • the network card writing the target data to the data buffer area includes:
  • the network card searches for a data queue corresponding to the target UDP port number, and the target UDP port number and the at least one data queue are pre-established with a one-to-one mapping relationship;
  • the network card searches for a corresponding entry from the data queue according to a write index, and an address of the sub-area is stored in the entry;
  • the network card writes the target data into a sub-region corresponding to the entry, and adds 1 to an offset of the write index.
  • the processor determines whether an offset of the read index is less than an offset of the write index, and a start offset of the read index is the same as a start offset of the write index;
  • the processor performs the step of processing the target data in the sub-region pointed by the index, and increments the offset of the read index by one.
  • an index for guiding the used entry that is, reading the index
  • the start offset of the read index is the same as the start offset of the write index.
  • the offset of the read index is smaller than the offset of the write index by one.
  • the processor determines that the offset of the read index is less than the offset of the write index, it indicates that there is new data write in the data buffer.
  • the processor finds the data cache according to the entry pointed to by the read index. Corresponding sub-area, and processing the target data in the sub-area, and adding 1 to the offset of the read index, so that the offset of the read index is again the same as the offset of the write index, that is, pointing to the same entry .
  • the processor determines After the offset to the read index is less than the offset of the write index, and the processor processes the target data buffered in the data buffer pointed to by the read index,
  • the method also includes:
  • the processor determines whether a client corresponding to the target data pointed by the read index establishes a connection with the server;
  • the target data is deleted.
  • the processor After the server is powered back on after a failure, if the processor finds that the data buffer has been written with target data that has not been processed. Then, the processor obtains the corresponding client information according to the data packet header of the target data, for example, the IP address of the client, and the processor can determine, according to the IP address, whether the client corresponding to the IP address establishes a connection with the server, and if so, Then, the step of processing the target data pointed to by the read index is performed; if not, the target data is deleted.
  • the processor After the server is powered back on after a failure, if the processor finds that the data buffer has been written with target data that has not been processed. Then, the processor obtains the corresponding client information according to the data packet header of the target data, for example, the IP address of the client, and the processor can determine, according to the IP address, whether the client corresponding to the IP address establishes a connection with the server, and if so, Then, the step of processing the target data pointed to by the read
  • the target data includes: a data packet header and a data item
  • the processing, by the processor, the target data cached in the data buffer area pointed by the read index includes:
  • the processor performs a header processing on a data packet header of the target data buffered in the data buffer area pointed by the read index, and performs metadata processing on data items of the target data.
  • the processor processes the data packet header and the data item of the target data, and aims to record the basic information of the target data and the cache location information, and transmit the recorded information to the storage device for maintaining the storage.
  • the target data includes at least two data. a packet, the at least two data packets respectively carrying a sequence identifier;
  • Target data of the data buffer area includes:
  • the processor performs a connection process on the at least two data packets according to the sequence identifier.
  • the target data includes at least two data packets, at least two data packets respectively carry sequence identifiers, and the processor performs connection processing on multiple data packets according to the sequence identifier.
  • the client needs to split a large data packet into several small data packets (ie, segmented packets), and then send them to the server in turn, and cache.
  • segmented packets In multiple sub-areas of the data buffer.
  • the processor needs to associate the segmentation packets when processing the segmentation packets. To this end, the processor can perform connection processing on multiple segmented packets according to the sequence identifier carried in the data packet.
  • the processing, by the processor, the target data of the data buffer area includes:
  • the processor determines whether the target data is a segmentation packet
  • determining a packet type of the segmentation packet where the packet type includes: a preamble packet and a subsequent packet;
  • the processor searches for the subsequent packet of the preamble packet from the data buffer area;
  • segment packet is a subsequent packet, searching for the preamble packet of the subsequent packet from the data buffer area;
  • the data processing system further includes a storage device.
  • the method further includes: the processor writing the processed target data to the storage device.
  • the storage device functions as an external memory, and the processor writes the target data to the external storage after the processing of the target data is completed.
  • the network card receives the target data, and Before the network card writes the target data to the data buffer, the method further includes:
  • the network card parses the request type of the target data
  • the network card When the request type of the target data is a write request, the network card performs the step of writing the target data into the data buffer.
  • the request type of the target data may be divided into a write request and a read request, and the client may encapsulate the target data in different encapsulation manners according to different types of requests before sending the target data.
  • the target data for the write request can be in the A-package mode
  • the data packet for the read request can be in the B-package mode.
  • the NIC After receiving the target data, the NIC parses the request type of the target data according to the different manner of encapsulation of the target data.
  • the embodiment of the present invention processes only the target data of the write request, and the target data of the read request is not processed because the data requested by the target data is already stored on the server.
  • a second aspect of the embodiments of the present invention provides a data processing system, where the data processing system includes: a network card, a memory, and at least one processor, the network card shares a data buffer with the at least one processor, and the data cache The area is located in the memory;
  • the network card is configured to receive target data, and is configured to write the target data into the data buffer area;
  • the processor is configured to process target data of the data buffer.
  • the memory is a storage level memory SCM.
  • the data buffer area includes at least one sub-area, At least one sub-region is in one-to-one correspondence with the at least one processor;
  • the network card is further configured to receive an indication identifier, where the indication identifier is used to indicate a processor corresponding to the target data;
  • the network card is configured to write the target data into the data buffer area, where the network card is configured to write the target data into a sub-region corresponding to the processor indicated by the indication identifier;
  • the processor Processing, by the processor, the target data in the data buffer area, where the processor is configured to perform the target data in a sub-region corresponding to the processor indicated by the indication identifier Reason.
  • the target data includes at least two a data packet, where the at least two data packets respectively carry a sequence identifier
  • the processor is configured to process the target data of the data buffer area, where the processor is configured to perform connection processing on the at least two data packets according to the sequence identifier.
  • the data processing system further includes storing The device, after processing the target data of the data buffer, is further used to write the processed target data to the storage device.
  • the network card after receiving the target data, the network card performs unpacking and network protocol stack processing on the target data, and then the network card copies the processed target data to the shared data cache area of the network card and the processor.
  • the processor does not need to copy the target data from the data buffer to other data buffers, and only needs to process the target data in the data buffer, so compared with the prior art, In the embodiment of the invention, the number of copies of the target data is reduced, thereby reducing the occupation of the memory bandwidth of the system.
  • FIG. 1 is a schematic structural diagram of a storage node of a distributed storage system in the prior art
  • FIG. 2 is a schematic structural diagram of a storage node of a distributed storage system according to an embodiment of the present invention
  • FIG. 3 is a flowchart of steps of an embodiment of a data processing method according to an embodiment of the present invention.
  • Distributed file system can effectively solve the problem of data storage and management: a file system fixed at a certain location, extended to any number of locations / multiple file systems, a large number of nodes constitute a file system network. Each node can be distributed in different locations, and communication and data transmission between nodes can be performed through the network. When people use a distributed file system, they don't need to care which node the data is stored on, or from which node it gets. It only needs to manage and store the data in the file system as if it were a local file system.
  • the target data in the first data buffer area 1021 is copied into the second data buffer area 1022, which is bound to occupy the system.
  • Memory bandwidth is a bottleneck in the system, and each copy of the data will consume the system's memory bandwidth.
  • the first data buffer area 1021 corresponding to the network card 101 and the second data buffer area 1022 corresponding to the processor 103 are used. Merging, so that the network card 101 and the processor 103 share data Storage area 2021.
  • the network card 201 After receiving the target data, the network card 201 performs unpacking and network protocol stack processing on the target data, and then the network card 201 copies the processed target data to the network card 201 and the processor 203 to share the data buffer area 2021, and the processor 203 processes
  • the target data does not need to be copied from the data buffer 2021 to other data buffers, and the target data only needs to be processed in the data buffer 2021, thereby reducing the number of times of copying the target data. , thereby reducing the use of the system's memory bandwidth.
  • FIG. 2 is a schematic structural diagram of a storage node of a distributed storage system according to an embodiment of the present invention.
  • the data processing system may be a multi-processor system, such as a distributed storage system, a distributed file system, and a key-value KV system. Etc., here is not limited.
  • the embodiment of the present invention is described in detail by taking a distributed storage system as an example.
  • the server of the storage node includes:
  • the network card 201, the memory 202, and the processor 203 are connected through the I/O bus 204 and the memory bus 205.
  • the network card 201 and the processor 203 share the data buffer area 2021, and the data buffer area 2021 is located in the memory 202.
  • the network card 201 is preferably a multi-queue network card with a direct memory access DMA engine. Among them, the DMA engine is used to allocate DMA channels (data queues) to the target data.
  • the request type of the target data may be divided into a write request and a read request, wherein the write request refers to storing the data in the target data sent by the client on the server of the corresponding storage node; the read request refers to the target data sent according to the client.
  • the information carried in the information is obtained from the server corresponding to the storage node. Essentially, the difference between the write request and the target data of the read request is whether the data packet carries data stored on the server.
  • the first data buffer 1021 and the second data buffer 1022 in the existing memory 202 are merged to form a shared data buffer 2021, so that the network card 201 shares the data buffer 2021 with the processor 203.
  • the network card 201 After receiving the target data, the network card 201 performs unpacking and network protocol stack processing on the target data, and then the network card 201 copies the processed target data to the network card 201 and the processor 203 to share the data buffer 2021. That is to say, the process of receiving the packet by the network card 201 is the process of data storage.
  • the processor 203 also performs in the data buffer 2021 without copying the target data into other data buffers, thereby reducing the number of times of copying the target data, thereby reducing the pair.
  • the memory bandwidth of the system is occupied.
  • the data buffer area 2021 may be further divided into a plurality of sub-areas, each of which is further divided into a plurality of sub-areas.
  • the sub-area corresponds to a processor 203, and the target data that needs to be specified by the processor 203 can be directly written by the network card 201 to the sub-area corresponding to the designated processor 203.
  • the memory 202 may preferably be a storage level memory SCM (non-volatile memory), and the SCM can ensure that the data is not lost if the server is powered off.
  • SCM storage level memory
  • the SCM can be used as the memory of the system or as the external storage of the system, which is not limited herein.
  • FIG. 3 is a flowchart of steps in an embodiment of a data processing method according to an embodiment of the present invention.
  • the data processing system may be a distributed storage system, a distributed file system, a key value KV system, etc., which is not limited herein.
  • the method includes:
  • the network card receives the target data.
  • the data buffer area may be divided into multiple sub-areas, wherein the plurality of sub-areas are in one-to-one correspondence with the plurality of processors, and the network card further receives an indication identifier, where the indication identifier is used to indicate the target data.
  • the corresponding processor that is, when the client sends the target data to the server, may specify which processor to process the target data, or indicate which sub-area the network card writes the target data to.
  • the server can inform the client processor (or sub-area) of the mapping relationship with the user datagram protocol UDP port number through a message protocol.
  • the client sends the target data, it sends the target UDP port number together, so that the target data can be allocated to the processor (or sub-area) corresponding to the target UDP port number.
  • the indication identifier may be the foregoing UDP port number, or may be a physical port IP or other indication identifier
  • the target data may be in the same data packet as the indication identifier, uniformly received by the network card, or not in the same data packet, by the network card.
  • the network card writes the target data into the data buffer area.
  • the network card After receiving the target data, the network card performs unpacking and network protocol stack processing on the target data, and writes the target data into the data buffer. More specifically, in the multi-processor system, the network card further receives the indication identifier, and writes the target data into the sub-region corresponding to the processor indicated by the indication identifier according to the indication identifier, so that the processor indicating the indication indication is The target data is processed.
  • the method may further include the following steps:
  • the network card When the request type of the target data is a write request, the network card performs the step of writing the target data into the data buffer.
  • the request type of the target data can be divided into a write request and a read request.
  • the client encapsulates the target data in different encapsulation manners according to the type of the request.
  • the target data for the write request can be in the A-package mode
  • the data packet for the read request can be in the B-package mode.
  • the NIC parses the request type of the target data according to the different manner of encapsulation of the target data.
  • the embodiment of the present invention processes only the target data of the write request, and the target data of the read request is not processed because the data requested by the target data is already stored on the server. Therefore, the target data of the read request is not explained in the embodiment of the present invention.
  • the network card determines that the request type of the target data is a write request, the network card performs the step of writing the target data into the data buffer.
  • the foregoing step 302 may further include:
  • Step 1 The network card obtains the target UDP port number from the target data.
  • Step 2 The network card searches for a data queue corresponding to the target UDP port number, and the target UDP port number and the data queue are pre-established in a one-to-one correspondence relationship;
  • Step 3 The network card searches for the corresponding entry from the data queue according to the write index, and the address of the sub-area is stored in the entry;
  • Step 4 The network card writes the target data into the sub-region corresponding to the entry, and offsets the write index. Add 1 to the quantity.
  • an index for directing the available entries may be specifically set, that is, the index is written.
  • step four since the address pointed to by the write index stores the address of the sub-area of the data buffer, the DMA engine only needs to write the target data into the sub-area corresponding to the address. At the same time, the offset of the write index is incremented by 1, so that the write index points to the next available entry.
  • the at least one processor processes the target data of the data buffer.
  • the data buffer includes a plurality of sub-regions, each sub-region corresponding to one processor, and the processor processes the target data in the corresponding sub-region.
  • the processor performs connection processing on multiple data packets according to the sequence identifier.
  • the client needs to split a large data packet into several small data packets (ie, segmented packets), and then send them to the server in turn, and cache.
  • segmented packets In the data cache.
  • the processor processes the segmentation packets, the segmentation packages need to be associated. To this end, the processor can perform connection processing on multiple segmented packets according to the sequence identifier carried in the data packet.
  • the target data may include at least two data packets, and at least two data packets respectively carry sequence identifiers.
  • the processing of the target data of the data buffer by the processor may further include:
  • the processor performs connection processing on the at least two data packets according to the sequence identifier.
  • the sequence identifier may be a pointer that is set at the front end or the tail end of the packet for connecting two adjacent data packets.
  • processing the target data of the data buffer by the processor may further include:
  • the processor determines whether the target data is a segmentation packet
  • the packet type of the segmentation packet is determined, and the packet type includes: a preamble packet and a subsequent packet;
  • the processor searches the data buffer area for a subsequent packet of the preamble packet;
  • the fragmented packet is a subsequent packet, it is searched from the data buffer whether there is a preamble packet of the subsequent packet;
  • the data processing system may further include a storage device.
  • the method may further include the following steps:
  • the processor writes the processed target data to the storage device.
  • the storage device functions as an external storage, and after the processor finishes processing the target data, the target data is written to the external storage.
  • An embodiment of the present invention is described by taking a processor as an example.
  • the processor does not need to copy the target data from the data buffer to other data buffers, and only needs to process the data in the data buffer.
  • the target data is sufficient to reduce the number of copies of the target data, thereby reducing the memory bandwidth of the system.
  • the target data mainly includes: a packet header and a data item.
  • the data packet header mainly records the client information, such as a mac header, an ip header, a tcp header, etc., and the data entry records the data that the client needs to write into the server or needs to be from the server. Get the address information of the data.
  • the step of instructing the processor indicated by the identifier to process the target data of the data buffer may include:
  • the processor indicating the indication indication performs packet processing on the packet header of the target data of the data buffer area, and performs metadata processing on the data item of the target data.
  • the processor processes the data packet header and the data item of the target data, and aims to record the basic information of the target data and the cache location information, and transmit the recorded information to the upper layer system for maintaining the upper layer.
  • a mapping table between the system and the processor about the location of the target data in the data buffer, so that the upper layer system can call the target data according to the mapping table when the target data needs to be used.
  • the processor can process the content of the target data itself after processing the basic information of the target data and the cache location information, which is not limited herein.
  • the method may further include the following steps:
  • the processor determines whether the offset of the read index is less than the offset of the write index, and the start offset of the read index The amount is the same as the starting offset of the write index;
  • the processor performs the step of processing the target data in the sub-region pointed to by the index and increments the offset of the read index by one.
  • the NIC After the NIC adds 1 to the offset of the write index, the corresponding triggers the processor to determine the offset of the read index and the write index.
  • an index for guiding the used entry that is, reading the index
  • the start offset of the read index is the same as the start offset of the write index.
  • the offset of the read index is smaller than the offset of the write index by one.
  • the processor determines that the offset of the read index is less than the offset of the write index, it indicates that there is new data write in the data buffer.
  • the processor finds the data cache according to the entry pointed to by the read index. Corresponding sub-area, and processing the target data in the sub-area, and adding 1 to the offset of the read index, so that the offset of the read index is again the same as the offset of the write index, that is, pointing to the same entry .
  • the processor determines that the offset of the read index is less than the offset of the write index, and the processor caches the target data in the data buffer pointed to by the read index.
  • the method may further include the steps before processing:
  • the processor determines whether the client corresponding to the target data pointed by the read index establishes a connection with the server;
  • the processor After the server is powered back on after a failure, if the processor finds that the target data has not been processed in the data buffer. Then, the processor obtains the corresponding client information according to the data packet header of the target data, for example, the IP address of the client, and the processor can determine, according to the IP address, whether the client corresponding to the IP address establishes a connection with the server, and if so, The step of processing the target data pointed to by the read index is performed; if not, the target data is deleted. That is, if the client corresponding to the target data is disconnected from the server, the processor deletes the cache that has been cached in the data cache, but Target data that has not been processed.
  • the processor finishes processing the target data, it also feeds back a confirmation message ACK to the client.
  • the processor can eliminate part of the data in the data buffer to obtain enough space to cache other target data.
  • the disclosed system, apparatus, and method may be implemented in other manners.
  • the device embodiments described above are merely illustrative.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.
  • the integrated unit if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium.
  • the technical solution of the present invention which is essential or contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product stored in a storage medium. , including a number of instructions to make a computer device (which can be a personal computer, a server, Or a network device or the like) performing all or part of the steps of the method of the various embodiments of the present invention.
  • the foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and the like. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

一种数据处理方法及系统，所述方法应用于数据处理系统，所述数据处理系统包括网卡（201）、内存（202）和至少一个处理器（203），所述网卡（201）与所述至少一个处理器（203）共享数据缓存区（2021），所述数据缓存区（2021）位于所述内存（202）中；所述方法包括：所述网卡接收目标数据（301）；所述网卡将所述目标数据写入所述数据缓存区（302）；所述至少一个处理器对所述数据缓存区的目标数据进行处理（303）。该方法及系统旨在减少目标数据的拷贝次数，以降低对系统的内存带宽的占用。

Description

一种数据处理方法及系统 技术领域
本发明涉及数据处理系统技术领域，尤其涉及一种数据处理方法及系统。
背景技术
随着移动设备、社交网络、互联网等多种应用的蓬勃发展，人类社会产生的数据呈爆炸式增长。为了满足以海量数据为基础的数据密集型应用的存储要求，分布式存储系统应运而生。
分布式存储系统是将数据分散地存储在多个独立的存储服务器上，采用可扩展的系统结构，利用多台存储服务器分担存储负荷，它不但提高了系统的可靠性、可用性和存取效率，还易于扩展。
请参阅图1，图1为现有技术中分布式存储系统一个存储节点的架构示意图，该存储节点的服务器包括：网卡101、内存102、处理器103、I/O总线104以及内存总线105，其中，网卡101、内存102和处理器103通过I/O总线104和内存总线105连接。
网卡101对应有第一数据缓存区1021，处理器103对应有第二数据缓存区1022，第一数据缓存区1021和第二数据缓存区1022均位于内存102中，且两者通过内存总线105连接。
具体地，当客户端向服务器发送数据包时，网卡101接收到目标数据之后，便对目标数据进行解包和网络协议栈处理，之后网卡101将处理后的目标数据拷贝到第一数据缓存区1021中，同时触发中断，驱动程序通过中断将第一数据缓存区1021中的目标数据拷贝到第二数据缓存区1022中，以供处理器103处理。
由于第一数据缓存区1021与第二数据缓存区1022是通过内存总线105连接的，那么要将第一数据缓存区1021中的目标数据拷贝到第二数据缓存区1022中，势必会占用系统的内存带宽。内存带宽是系统的一个瓶颈，每一次的数据拷贝都将会消耗系统的内存带宽。
发明内容
本发明实施例提供了一种数据处理方法及系统，旨在减少目标数据的拷贝次数，以降低对系统的内存带宽的占用。
本发明实施例的第一方面提供一种数据处理方法，所述方法应用于数据处理系统，所述数据处理系统包括网卡、内存和至少一个处理器，所述网卡与所述至少一个处理器共享数据缓存区，所述数据缓存区位于所述内存中；
所述方法包括：
所述网卡接收目标数据；
所述网卡将所述目标数据写入所述数据缓存区；
所述至少一个处理器对所述数据缓存区的目标数据进行处理。
在上述方法中，网卡接收到目标数据之后，便对目标数据进行解包和网络协议栈处理，之后网卡将处理后的目标数据缓存在网卡与处理器共享数据缓存区中，处理器在处理该目标数据时，无需将该目标数据从该数据缓存区中拷贝到其它数据缓存区，只需在该数据缓存区中处理该目标数据即可，以此减少对该目标数据的拷贝次数，从而降低对系统的内存带宽的占用。
结合本发明实施例的第一方面，在本发明实施例的第一方面的第一种实现方式中，所述内存为存储级内存SCM。
在上述方法中，内存采用非易失性存储器SCM，可以使目标数据得到持久化的存储。另外，SCM既可以作为系统的内存，也可以作为系统的外存，具体此处不做限定。
结合本发明实施例的第一方面或第一方面的第一实现方式，在本发明实施例的第一方面的第二种实现方式中，所述数据缓存区包含至少一个子区域，所述至少一个子区域与所述至少一个处理器一一对应；
所述方法还包括：所述网卡接收指示标识，所述指示标识用于指示所述目标数据对应的处理器；
所述网卡将所述目标数据写入所述数据缓存区包括：
所述网卡将所述目标数据写入所述指示标识指示的处理器对应的子区域；
所述至少一个处理器对所述数据缓存区的目标数据进行处理包括：
所述指示标识指示的处理器对与其对应的子区域中的所述目标数据进行处理。
在上述方法中，数据缓存区可以被分为多个子区域，其中，多个子区域与多个处理器一一对应，网卡还接收指示标识，该指示标识用于指示目标数据对应的处理器，即客户端向服务器发送目标数据时，可以指定由哪一个处理器来对目标数据进行处理，或者指示网卡将目标数据写入哪一个子区域。
可选地，所述网卡将所述目标数据写入所述数据缓存区包括：
所述网卡从所述目标数据中获取目标用户数据报协议UDP端口号；
所述网卡查找所述目标UDP端口号对应的数据队列，所述目标UDP端口号与所述至少一个数据队列预先建立有一一对应的映射关系；
所述网卡根据写索引从所述数据队列中查找对应的项entry，所述子区域的地址存放于所述entry中；
所述网卡将所述目标数据写入所述entry对应的子区域中，并将所述写索引的偏移量加1。
所述处理器判断读索引的偏移量是否小于所述写索引的偏移量，所述读索引的起始偏移量与所述写索引的起始偏移量相同；
若是，则所述处理器执行对所述索引指向的子区域中的所述目标数据进行处理的步骤，并将所述读索引的偏移量加1。
为便于处理器能够及时的了解数据缓存区中写入数据的情况，为此，可专门设置用于指引已用entry的索引，即读索引。其中，读索引的起始偏移量与写索引的起始偏移量相同。当网卡在引导目标数据写入数据缓存区对应的子区域中后，将写索引的偏移量加1，那么此时读索引的偏移量要比写索引的偏移量小1。当处理器判断到读索引的偏移量比写索引的偏移量小1，说明数据缓存区中有新的数据写入，此时，处理器会根据读索引指向的entry找到数据缓存区中对应的子区域，并对该子区域中的目标数据进行处理，同时将读索引的偏移量加1，使得读索引的偏移量与写索引的偏移量再次回归相同，即指向同一entry。
可选地，当所述数据处理系统的服务器发生故障重启时，所述处理器判断 到所述读索引的偏移量小于所述写索引的偏移量之后，且所述处理器对所述读索引指向的所述数据缓存区中缓存的所述目标数据进行处理之前，所述方法还包括：
所述处理器判断所述读索引指向的所述目标数据所对应的客户端是否与所述服务器建立连接；
若是，则执行对所述处理器执行对所述索引指向的子区域中的所述目标数据进行处理的步骤；
若否，则删除所述目标数据。
服务器在发生故障重新上电后，如果处理器发现数据缓存区中写入有尚未进行处理的目标数据。那么，处理器会根据目标数据的数据包头来获取对应客户端信息，如：客户端的IP地址，处理器可根据该IP地址判断该IP地址所对应的客户端是否与服务器建立连接，如果是，则执行对读索引指向的目标数据进行处理的步骤；如果否，则删除目标数据。
可选地，所述目标数据包括：数据包头和数据项，所述处理器对所述读索引指向的所述数据缓存区中缓存的所述目标数据进行处理包括：
所述处理器对所述读索引指向的所述数据缓存区中缓存的所述目标数据的数据包头进行包头处理，并对所述目标数据的数据项进行元数据处理。
在上述方法中，处理器对目标数据的数据包头以及数据项进行处理，旨在对该目标数据的基本信息以及缓存位置信息进行记录，并将该记录的信息传递给存储设备，用于维护存储设备与处理器之间关于目标数据在数据缓存区中位置的映射表，以便于存储设备在需要调用该目标数据时，处理器可以及时根据该映射表调用该目标数据。
结合本发明实施例的第一方面至第一方面的第二实现方式中的任意一种，在本发明实施例的第一方面的第三种实现方式中，所述目标数据包含至少两个数据包，所述至少两个数据包分别携带序列标识；
所述处理器对所述数据缓存区的目标数据进行处理包括：
所述处理器根据所述序列标识，对所述至少两个数据包进行连接处理。
在上述方法中，如果目标数据的大小大于最大传输单元MTU的大小，在 对目标数据进行传输时，需要对其进行拆包传输，即目标数据包含至少两个数据包，至少两个数据包分别携带序列标识，处理器根据序列标识，对多个数据包进行连接处理。
当客户端要发送给服务器的数据大于网卡的MTU时，则客户端需要采用将一个大的数据包拆分成若干个小的数据包(即分段包)，然后依次发送给服务器，并缓存在数据缓存区的多个子区域中。针对分段包，虽然是缓存在数据缓存区不同的子区域中，但是处理器在对该些分段包进行处理时，需要将该些分段包联系起来。为此，处理器可根据数据包中携带的序列标识对多个分段包进行连接处理。
可选地，所述处理器对所述数据缓存区的目标数据进行处理包括：
所述处理器判断所述目标数据是否为分段包；
若是，则确定所述分段包的包类型，所述包类型包括：前序包和后续包；
当所述分段包为前序包时，则所述处理器从所述数据缓存区中查找是否有所述前序包的后续包；
若有，则将所述后续包与所述前序包进行关联；
或者，
当所述分段包为后续包时，则从所述数据缓存区中查找是否有所述后续包的前序包；
若有，则将所述前序包与所述后续包进行关联。
结合本发明实施例的第一方面至第一方面的第三实现方式中的任意一种，在本发明实施例的第一方面的第四种实现方式中，所述数据处理系统还包括存储设备，所述处理器对所述数据缓存区的目标数据进行处理之后，所述方法还包括：所述处理器将处理后的目标数据写入所述存储设备。
在上述方法中，该存储设备充当外存的功能，处理器在对目标数据完成处理之后，会将目标数据写入外存。
结合本发明实施例的第一方面至第一方面的第四实现方式中的任意一种，在本发明实施例的第一方面的第五种实现方式中，所述网卡接收目标数据之后，且所述网卡将所述目标数据写入所述数据缓存区之前，所述方法还包括：
所述网卡解析所述目标数据的请求类型；
当所述目标数据的请求类型为写请求时，则所述网卡执行将所述目标数据写入所述数据缓存区的步骤。
在上述方法中，目标数据的请求类型可分为写请求和读请求，客户端在发送目标数据之前，会根据请求类型的不同，采用不同的封装方式对该目标数据进行封装。如：针对写请求的目标数据可以采用A封装方式，针对读请求的数据包可以采用B封装方式。
网卡在收到目标数据之后，会根据目标数据的封装方式的不同，解析出该目标数据的请求类型。本发明实施例仅对写请求的目标数据做出处理，而对于读请求的目标数据，由于目标数据所请求的数据已经存储于服务器上的，本发明实施例不做处理。
本发明实施例第二方面提供了一种数据处理系统，所述数据处理系统包括：网卡、内存和至少一个处理器，所述网卡与所述至少一个处理器共享数据缓存区，所述数据缓存区位于所述内存中；
所述网卡用于接收目标数据，并用于将所述目标数据写入所述数据缓存区；
所述处理器用于对所述数据缓存区的目标数据进行处理。
结合本发明实施例的第二方面，在本发明实施例的第二方面的第一种实现方式中，所述内存为存储级内存SCM。
结合本发明实施例的第二方面或第二方面的第一种实现方式，在本发明实施例的第二方面的第二种实现方式中，所述数据缓存区包含至少一个子区域，所述至少一个子区域与所述至少一个处理器一一对应；
所述网卡还用于接收指示标识，所述指示标识用于指示所述目标数据对应的处理器；
所述网卡用于将所述目标数据写入所述数据缓存区包括：所述网卡用于将所述目标数据写入所述指示标识指示的处理器对应的子区域；
所述处理器用于对所述数据缓存区的目标数据进行处理包括：所述处理器用于对与所述指示标识指示的处理器对应的子区域中的所述目标数据进行处 理。
结合本发明实施例的第二方面至第二方面的第二种实现方式中的任意一种，在本发明实施例的第二方面的第三种实现方式中，所述目标数据包含至少两个数据包，所述至少两个数据包分别携带序列标识；
所述处理器用于对所述数据缓存区的目标数据进行处理包括：所述处理器用于根据所述序列标识，对所述至少两个数据包进行连接处理。
结合本发明实施例的第二方面至第二方面的第三种实现方式中的任意一种，在本发明实施例的第二方面的第四种实现方式中，所述数据处理系统还包括存储设备，所述处理器对所述数据缓存区的目标数据进行处理之后，还用于将处理后的目标数据写入所述存储设备。
本发明实施例提供的技术方案中，网卡接收到目标数据之后，便对目标数据进行解包和网络协议栈处理，之后网卡将处理后的目标数据拷贝到网卡与处理器共享数据缓存区中，处理器在处理该目标数据时，无需将该目标数据从该数据缓存区中拷贝到其它数据缓存区，只需在该数据缓存区中处理该目标数据即可，因此相对于现有技术，本发明实施例中减少对该目标数据的拷贝次数，从而降低对系统的内存带宽的占用。
附图说明
图1为现有技术中分布式存储系统一个存储节点的架构示意图；
图2为本发明实施例中分布式存储系统一个存储节点的架构示意图；
图3为本发明实施例中数据处理方法一个实施例的步骤流程图。
具体实施方式
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第 三”“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
在信息爆炸的时代中，人们可以获取的数据成指数倍的增长，单纯通过增加硬盘个数来扩展计算机文件系统的存储容量的方式，在容量大小、容量增长速度、数据备份、数据安全等方面的表现都差强人意。分布式文件系统可以有效解决数据的存储和管理难题：将固定于某个地点的某个文件系统，扩展到任意多个地点/多个文件系统，众多的节点组成一个文件系统网络。每个节点可以分布在不同的地点，通过网络进行节点间的通信和数据传输。人们在使用分布式文件系统时，无需关心数据是存储在哪个节点上、或者是从哪个节点从获取的，只需要像使用本地文件系统一样管理和存储文件系统中的数据。
然而，随着分布式文件系统管理和存储的数据日趋庞大，数据在分布式文件系统中多个单元之间进行多次拷贝，消耗了大量内存带宽，而内存带宽是系统的一个瓶颈。如上述背景技术所述，当客户端向服务器发送数据包时，网卡101接收到目标数据之后，便对目标数据进行解包和网络协议栈处理，之后网卡101将处理后的目标数据拷贝到第一数据缓存区1021中，同时触发中断，驱动程序通过中断将第一数据缓存区1021中的目标数据拷贝到第二数据缓存区1022中。由于第一数据缓存区1021与第二数据缓存区1022是通过内存总线105连接的，那么第一数据缓存区1021中的目标数据要想拷贝到第二数据缓存区1022中，势必会占用系统的内存带宽。内存带宽是系统的一个瓶颈，每一次的数据拷贝都将会消耗系统的内存带宽。
本发明实施例，为减少上述目标数据在内存中的拷贝次数，以降低对系统的内存带宽的占用，将网卡101对应的第一数据缓存区1021与处理器103对应的第二数据缓存区1022进行合并，使得网卡101与处理器103共享数据缓 存区2021。网卡201接收到目标数据之后，便对目标数据进行解包和网络协议栈处理，之后网卡201将处理后的目标数据拷贝到网卡201与处理器203共享数据缓存区2021中，处理器203在处理该目标数据时，无需将该目标数据从该数据缓存区2021中拷贝到其它数据缓存区，只需在该数据缓存区2021中处理该目标数据即可，以此减少对该目标数据的拷贝次数，从而降低对系统的内存带宽的占用。
请参阅图2，图2为本发明实施例中分布式存储系统一个存储节点的架构示意图，数据处理系统可以为多处理器系统，如：分布式存储系统、分布式文件系统、键值KV系统等，具体此处不做限定。本发明实施例以分布式存储系统为例进行详述，如图2所示，该存储节点的服务器包括：
网卡201、内存202、至少一个处理器203(如：本发明实施例中以一个处理器203为例)、I/O总线204以及内存总线205。其中，网卡201、内存202和处理器203通过I/O总线204和内存总线205连接，网卡201和处理器203共享数据缓存区2021，且数据缓存区2021位于内存202中。
网卡201优选为多队列网卡，并带有直接内存存取DMA引擎。其中，DMA引擎用于为目标数据分配DMA通道(数据队列)。
目标数据的请求类型可分为写请求和读请求，其中，写请求是指将客户端发送的目标数据中的数据存储在对应存储节点的服务器上；读请求是指根据客户端发送的目标数据中携带的信息从对应存储节点的服务器上获取客户端所需要的数据。实质上，写请求与读请求的目标数据区别在于数据包中是否携带有存储在服务器上的数据。
将现有内存202中第一数据缓存区1021与第二数据缓存区1022合并形成共享数据缓存区2021，使得网卡201与处理器203共用数据缓存区2021。网卡201接收到目标数据之后，便对目标数据进行解包和网络协议栈处理，之后网卡201将处理后的目标数据拷贝到网卡201与处理器203共享数据缓存区2021中。也就是说，网卡201收包的过程即为数据存储的过程。处理器203在处理该目标数据时，也是在该数据缓存区2021中进行，而无需将该目标数据拷贝到其它数据缓存区中，以此减少对该目标数据的拷贝次数，从而降低对 系统的内存带宽的占用。
本发明实施例将第一数据缓存区1021与第二数据缓存区1022进行合并形成共享数据缓存区2021时，在多处理器系统中，还可以进一步将数据缓存区2021划分为多个子区域，每个子区域对应一个处理器203，需要指定处理器203处理的目标数据可以由网卡201直接写入指定的处理器203对应的子区域。
为了使目标数据能够持久化的得到存储，内存202可以优选为存储级内存SCM(非易失性存储器)，SCM可以保证数据在服务器掉电的情况下不丢失。另外，SCM既可以作为系统的内存，也可以作为系统的外存，具体此处不做限定。
上面对本发明实施例中的数据处理系统进行了描述，下面对本发明实施例中的数据处理方法进行描述，请参阅图3，图3为本发明实施例中数据处理方法一个实施例的步骤流程图，应用于数据处理系统，该数据处理系统可以是分布式存储系统、分布式文件系统、键值KV系统等，具体此处不做限定。如图3所示，该方法包括：
301、网卡接收目标数据；
可选的，在多处理器系统中，数据缓存区可以被分为多个子区域，其中，多个子区域与多个处理器一一对应，网卡还接收指示标识，该指示标识用于指示目标数据对应的处理器，即客户端向服务器发送目标数据时，可以指定由哪一个处理器来对目标数据进行处理，或者指示网卡将目标数据写入哪一个子区域。
例如：在客户端与服务器建立连接时，服务器可以通过消息协议告知客户端处理器(或子区域)与用户数据报协议UDP端口号的映射关系。客户端在发送目标数据时，会一起发送目标UDP端口号，使得该目标数据能够分配到该目标UDP端口号对应的处理器(或子区域)。
应理解，该指示标识可以是上述UDP端口号，也可以是物理端口IP或其他指示标识，且目标数据可以与指示标识处于相同数据包，由网卡统一接收，或不处于相同数据包，由网卡分别接收，本发明实施例对此并对指示标识的形 式，以及网卡接收指示标识的形式进行限定。
302、网卡将目标数据写入数据缓存区；
网卡接收到目标数据之后，便对目标数据进行解包和网络协议栈处理，并将目标数据写入数据缓存区。更具体的，在多处理器系统中，网卡还接收指示标识，并根据该指示标识将目标数据写入该指示标识指示的处理器对应的子区域中，以供指示标识指示的处理器对该目标数据进行处理。
可选地，在步骤302之前，还可以包括步骤：
网卡解析目标数据的请求类型；
当目标数据的请求类型为写请求时，则网卡执行将目标数据写入数据缓存区的步骤。
目标数据的请求类型可分为写请求和读请求，客户端在发送目标数据之前，会根据请求类型的不同，采用不同的封装方式对该目标数据进行封装。如：针对写请求的目标数据可以采用A封装方式，针对读请求的数据包可以采用B封装方式。网卡在收到目标数据之后，会根据目标数据的封装方式的不同，解析出该目标数据的请求类型。
本发明实施例仅对写请求的目标数据做出处理，而对于读请求的目标数据，由于目标数据所请求的数据已经存储于服务器上的，本发明实施例不做处理。因此，对于读请求的目标数据，本发明实施例不做阐释。当网卡确定了该目标数据的请求类型为写请求的情况下，网卡才执行将该目标数据写入数据缓存区中的步骤。
可选地，如：上述指示标识为服务器的UDP端口号，上述步骤302具体还可以包括：
步骤一：网卡从目标数据中获取目标UDP端口号；
步骤二：网卡查找目标UDP端口号对应的数据队列，目标UDP端口号与数据队列预先建立有一一对应的映射关系；
步骤三：网卡根据写索引从数据队列中查找对应的项entry，子区域的地址存放于entry中；
步骤四：网卡将目标数据写入entry对应的子区域中，并将写索引的偏移 量加1。
在步骤三中，为便于DMA引擎找到数据队列中可用entry，可专门设置用于指引可用entry的索引，即写索引。
在步骤四中，由于写索引指向的entry中存放有数据缓存区的子区域的地址，所以DMA引擎只需将目标数据写入该地址对应的子区域中即可。同时将写索引的偏移量加1，使得该写索引指向下一个可用entry。
303、至少一个处理器对数据缓存区的目标数据进行处理。
具体的，在多处理器系统中，数据缓存区包含多个子区域，每一个子区域对应一个处理器，处理器对与其对应的子区域中的目标数据进行处理。
更具体的，如果目标数据的大小大于最大传输单元MTU的大小，在对目标数据进行传输时，需要对其进行拆包传输，即目标数据包含至少两个数据包，至少两个数据包分别携带序列标识，处理器根据序列标识，对多个数据包进行连接处理。
当客户端要发送给服务器的数据大于网卡的MTU时，则客户端需要采用将一个大的数据包拆分成若干个小的数据包(即分段包)，然后依次发送给服务器，并缓存在数据缓存区中。处理器在对该些分段包进行处理时，需要将该些分段包联系起来。为此，处理器可根据数据包中携带的序列标识对多个分段包进行连接处理。
本发明实施例，该目标数据可以包含至少两个数据包，至少两个数据包分别携带序列标识。为此，处理器对数据缓存区的目标数据进行处理还可以包括：
处理器根据序列标识，对该至少两个数据包进行连接处理。
该序列标识可以是设置在数据包前端或尾端用于连接相邻的两个数据包的指针。
可选地，处理器对数据缓存区的目标数据进行处理还可以包括：
处理器判断目标数据是否为分段包；
若是，则确定分段包的包类型，包类型包括：前序包和后续包；
当分段包为前序包时，则处理器从数据缓存区中查找是否有前序包的后续包；
若有，则将后续包与前序包进行关联；
或者，
当分段包为后续包时，则从数据缓存区中查找是否有后续包的前序包；
若有，则将前序包与后续包进行关联。
可选地，数据处理系统还可以包括存储设备，处理器对数据缓存区的目标数据进行处理之后，该方法还可以包括步骤：
处理器将处理后的目标数据写入存储设备。
在本发明实施例中，该存储设备充当外存的功能，处理器在对目标数据完成处理之后，会将目标数据写入外存。
本发明实施例以一个处理器为例进行说明，处理器在处理该目标数据时，无需将该目标数据从该数据缓存区中拷贝到其它数据缓存区，只需在该数据缓存区中处理该目标数据即可，以此减少对该目标数据的拷贝次数，从而降低对系统的内存带宽的占用。
目标数据主要包括：数据包头和数据项，数据包头主要记载的是客户端信息，如：mac头、ip头、tcp头等，数据项记载的是客户端需要写入服务器中的数据或者需要从服务器中获取数据的地址信息。指示标识指示的处理器对数据缓存区的目标数据进行处理的步骤可以包括：
指示标识指示的处理器对数据缓存区的目标数据的数据包头进行包处理，并对目标数据的数据项进行元数据处理。
本发明实施例，处理器对目标数据的数据包头以及数据项进行处理，旨在对该目标数据的基本信息以及缓存位置信息进行记录，并将该记录的信息传递给上层系统，用于维护上层系统与处理器之间关于目标数据在数据缓存区中位置的映射表，以便于上层系统在需要使用该目标数据时，处理器可以及时根据该映射表调用该目标数据。
当然，处理器除了对目标数据的基本信息以及缓存位置信息进行处理之后，还可以对目标数据本身内容进行处理，具体此处不做限定。
可选地，在步骤303之前，还可以包括步骤：
处理器判断读索引的偏移量是否小于写索引的偏移量，读索引的起始偏移 量与写索引的起始偏移量相同；
若是，则处理器执行对索引指向的子区域中的目标数据进行处理的步骤，并将读索引的偏移量加1。
网卡将写索引的偏移量加1之后，相应的会触发处理器去判断读索引与写索引的偏移量的大小。
为便于处理器能够及时的了解数据缓存区中写入数据的情况，为此，可专门设置用于指引已用entry的索引，即读索引。其中，读索引的起始偏移量与写索引的起始偏移量相同。当网卡在引导目标数据写入数据缓存区对应的子区域中后，将写索引的偏移量加1，那么此时读索引的偏移量要比写索引的偏移量小1。当处理器判断到读索引的偏移量比写索引的偏移量小1，说明数据缓存区中有新的数据写入，此时，处理器会根据读索引指向的entry找到数据缓存区中对应的子区域，并对该子区域中的目标数据进行处理，同时将读索引的偏移量加1，使得读索引的偏移量与写索引的偏移量再次回归相同，即指向同一entry。
可选地，当数据处理系统的服务器发生故障重启时，处理器判断到读索引的偏移量小于写索引的偏移量之后，且处理器对读索引指向的数据缓存区中缓存的目标数据进行处理之前，该方法还可以包括步骤：
处理器判断读索引指向的目标数据所对应的客户端是否与服务器建立连接；
若是，则执行对处理器执行对索引指向的子区域中的目标数据进行处理的步骤；
若否，则删除目标数据。
服务器在发生故障重新上电后，如果处理器发现到数据缓存区中写入有尚未进行处理的目标数据。那么，处理器会根据目标数据的数据包头来获取对应客户端信息，如：客户端的IP地址，处理器可根据该IP地址判断该IP地址对应的客户端是否与服务器建立连接，如果是，则执行对读索引指向的目标数据进行处理的步骤；如果否，则删除目标数据。也就是说，如果目标数据对应的客户端与服务器断开了连接，则处理器会删除已经缓存在数据缓存区中，但 尚未进行处理的目标数据。
另外，处理器在对目标数据完成处理之后，相应的，还会向客户端反馈一份确认消息ACK。
需要说明的是，当数据缓存区出现缓存空间不足时，则处理器可以将数据缓存区中的部分数据进行淘汰，以获得足够的空间来缓存后续其它目标数据。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统，装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器， 或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，以上实施例仅用以说明本发明的技术方案，而非对其限制；尽管参照前述实施例对本发明进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本发明各实施例技术方案的精神和范围。

Claims (10)

  1. 一种数据处理方法，其特征在于，所述方法应用于数据处理系统，所述数据处理系统包括网卡、内存和至少一个处理器，所述网卡与所述至少一个处理器共享数据缓存区，所述数据缓存区位于所述内存中；
    所述方法包括：
    所述网卡接收目标数据；
    所述网卡将所述目标数据写入所述数据缓存区；
    所述至少一个处理器对所述数据缓存区的目标数据进行处理。
  2. 根据权利要求1所述的方法，其特征在于，所述内存为存储级内存SCM。
  3. 根据权利要求1或2所述的方法，其特征在于，所述数据缓存区包含至少一个子区域，所述至少一个子区域与所述至少一个处理器一一对应；
    所述方法还包括：所述网卡接收指示标识，所述指示标识用于指示所述目标数据对应的处理器；
    所述网卡将所述目标数据写入所述数据缓存区包括：所述网卡将所述目标数据写入所述指示标识指示的处理器对应的子区域；
    所述至少一个处理器对所述数据缓存区的目标数据进行处理包括：
    所述指示标识指示的处理器对与其对应的子区域中的所述目标数据进行处理。
  4. 根据权利要求1-3任一项所述的方法，其特征在于，所述目标数据包含至少两个数据包，所述至少两个数据包分别携带序列标识；
    所述处理器对所述数据缓存区的目标数据进行处理包括：
    所述处理器根据所述序列标识，对所述至少两个数据包进行连接处理。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述数据处理系统还包括存储设备，所述处理器对所述数据缓存区的目标数据进行处理之后，所述方法还包括：所述处理器将处理后的目标数据写入所述存储设备。
  6. 一种数据处理系统，其特征在于，所述数据处理系统包括：网卡、内存和至少一个处理器，所述网卡与所述至少一个处理器共享数据缓存区，所述数据缓存区位于所述内存中；
    所述网卡用于接收目标数据，并用于将所述目标数据写入所述数据缓存区；
    所述处理器用于对所述数据缓存区的目标数据进行处理。
  7. 根据权利要求6所述的系统，其特征在于，所述内存为存储级内存SCM。
  8. 根据权利要求6或7所述的系统，其特征在于，所述数据缓存区包含至少一个子区域，所述至少一个子区域与所述至少一个处理器一一对应；
    所述网卡还用于接收指示标识，所述指示标识用于指示所述目标数据对应的处理器；
    所述网卡用于将所述目标数据写入所述数据缓存区包括：所述网卡用于将所述目标数据写入所述指示标识指示的处理器对应的子区域；
    所述处理器用于对所述数据缓存区的目标数据进行处理包括：所述处理器用于对与所述指示标识指示的处理器对应的子区域中的所述目标数据进行处理。
  9. 根据权利要求6-8任一项所述的系统，其特征在于，所述目标数据包含至少两个数据包，所述至少两个数据包分别携带序列标识；
    所述处理器用于对所述数据缓存区的目标数据进行处理包括：所述处理器用于根据所述序列标识，对所述至少两个数据包进行连接处理。
  10. 根据权利要求6-9任一项所述的系统，其特征在于，所述数据处理系统还包括存储设备，所述处理器对所述数据缓存区的目标数据进行处理之后，还用于将处理后的目标数据写入所述存储设备。
PCT/CN2016/079174 2016-04-13 2016-04-13 一种数据处理方法及系统 WO2017177400A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201680007359.9A CN107615259B (zh) 2016-04-13 2016-04-13 一种数据处理方法及系统
PCT/CN2016/079174 WO2017177400A1 (zh) 2016-04-13 2016-04-13 一种数据处理方法及系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/079174 WO2017177400A1 (zh) 2016-04-13 2016-04-13 一种数据处理方法及系统

Publications (1)

Publication Number Publication Date
WO2017177400A1 true WO2017177400A1 (zh) 2017-10-19

Family

ID=60042242

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/079174 WO2017177400A1 (zh) 2016-04-13 2016-04-13 一种数据处理方法及系统

Country Status (2)

Country Link
CN (1) CN107615259B (zh)
WO (1) WO2017177400A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115454900A (zh) * 2022-08-08 2022-12-09 北京阿帕科蓝科技有限公司 数据传输方法、装置、计算机设备、存储介质和程序产品

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113630341B (zh) * 2021-08-03 2023-06-06 武汉绿色网络信息服务有限责任公司 数据信息处理的方法和服务器

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7533154B1 (en) * 2004-02-04 2009-05-12 Advanced Micro Devices, Inc. Descriptor management systems and methods for transferring data of multiple priorities between a host and a network
CN101616194A (zh) * 2009-07-23 2009-12-30 中国科学技术大学 主机网络性能优化系统及方法
CN101650698A (zh) * 2009-08-28 2010-02-17 曙光信息产业(北京)有限公司 直接存储器访问的实现方法
CN103942097A (zh) * 2014-04-10 2014-07-23 华为技术有限公司 一种数据处理方法、装置及具备相应装置的计算机
CN104580011A (zh) * 2013-10-23 2015-04-29 杭州华三通信技术有限公司 一种数据转发装置和方法
CN105335309A (zh) * 2014-05-30 2016-02-17 华为技术有限公司 一种数据传输方法及计算机

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060253662A1 (en) * 2005-05-03 2006-11-09 Bass Brian M Retry cancellation mechanism to enhance system performance
JP5491102B2 (ja) * 2009-08-27 2014-05-14 ルネサスエレクトロニクス株式会社 データプロセッサ
CN103763173B (zh) * 2013-12-31 2017-08-25 华为技术有限公司 数据传输方法和计算节点
US9632958B2 (en) * 2014-07-06 2017-04-25 Freescale Semiconductor, Inc. System for migrating stash transactions
CN104951412B (zh) * 2015-06-06 2018-01-02 华为技术有限公司 一种通过内存总线访问的存储装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7533154B1 (en) * 2004-02-04 2009-05-12 Advanced Micro Devices, Inc. Descriptor management systems and methods for transferring data of multiple priorities between a host and a network
CN101616194A (zh) * 2009-07-23 2009-12-30 中国科学技术大学 主机网络性能优化系统及方法
CN101650698A (zh) * 2009-08-28 2010-02-17 曙光信息产业(北京)有限公司 直接存储器访问的实现方法
CN104580011A (zh) * 2013-10-23 2015-04-29 杭州华三通信技术有限公司 一种数据转发装置和方法
CN103942097A (zh) * 2014-04-10 2014-07-23 华为技术有限公司 一种数据处理方法、装置及具备相应装置的计算机
CN105335309A (zh) * 2014-05-30 2016-02-17 华为技术有限公司 一种数据传输方法及计算机

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115454900A (zh) * 2022-08-08 2022-12-09 北京阿帕科蓝科技有限公司 数据传输方法、装置、计算机设备、存储介质和程序产品

Also Published As

Publication number Publication date
CN107615259A (zh) 2018-01-19
CN107615259B (zh) 2020-03-20

Similar Documents

Publication Publication Date Title
US10609150B2 (en) Lock management method in cluster, lock server, and client
WO2019127018A1 (zh) 存储系统访问方法及装置
US20240039995A1 (en) Data access system and method, device, and network adapter
KR20210075845A (ko) 네이티브 키-밸류 분산 스토리지 시스템
US11379405B2 (en) Internet small computer interface systems extension for remote direct memory access (RDMA) for distributed hyper-converged storage systems
US11314454B2 (en) Method and apparatus for managing storage device in storage system
WO2022007470A1 (zh) 一种数据传输的方法、芯片和设备
WO2016065611A1 (zh) 访问文件的方法、系统和主机
CN108540510B (zh) 一种云主机创建方法、装置及云服务系统
US20110153780A1 (en) Apparatus and methods for handling network file operations over a fibre channel network
US11321021B2 (en) Method and apparatus of managing mapping relationship between storage identifier and start address of queue of storage device corresponding to the storage identifier
WO2019134403A1 (zh) 发送数据包的方法、装置及计算机可读存储介质
CN109302448A (zh) 一种数据处理方法及装置
WO2016101662A1 (zh) 一种数据处理方法及相关服务器
CN111225003B (zh) 一种nfs节点配置方法和装置
WO2019000423A1 (zh) 一种数据存储方法及设备
WO2017177400A1 (zh) 一种数据处理方法及系统
US20230342087A1 (en) Data Access Method and Related Device
WO2016065610A1 (zh) 访问文件的方法、分布式存储系统和存储节点
US8090832B1 (en) Method and apparatus for allocating network protocol operation resources
WO2022073399A1 (zh) 存储节点、存储设备及网络芯片
WO2016065613A1 (zh) 访问文件的方法、分布式存储系统和网络设备
US11188394B2 (en) Technologies for synchronizing triggered operations
KR100785774B1 (ko) 객체 기반 파일 입출력 시스템 및 방법
WO2024051259A1 (zh) 数据处理方法及装置

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16898219

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 16898219

Country of ref document: EP

Kind code of ref document: A1