CN111756650A - Data processing method and device, operation chip and storage medium - Google Patents
Data processing method and device, operation chip and storage medium Download PDFInfo
- Publication number
- CN111756650A CN111756650A CN202010529663.8A CN202010529663A CN111756650A CN 111756650 A CN111756650 A CN 111756650A CN 202010529663 A CN202010529663 A CN 202010529663A CN 111756650 A CN111756650 A CN 111756650A
- Authority
- CN
- China
- Prior art keywords
- data
- processed
- chip
- data processing
- operation chip
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 27
- 238000011144 upstream manufacturing Methods 0.000 claims abstract description 115
- 238000012545 processing Methods 0.000 claims abstract description 102
- 238000000034 method Methods 0.000 claims abstract description 39
- 230000005540 biological transmission Effects 0.000 claims description 23
- 238000012544 monitoring process Methods 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 11
- 230000001133 acceleration Effects 0.000 abstract description 25
- 230000008569 process Effects 0.000 abstract description 22
- 230000009286 beneficial effect Effects 0.000 abstract description 2
- 230000009471 action Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003139 buffering effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/29—Flow control; Congestion control using a combination of thresholds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G06F15/163—Interprocessor communication
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/24—Traffic characterised by specific attributes, e.g. priority or QoS
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/76—Architectures of general purpose stored program computers
- G06F2015/761—Indexing scheme relating to architectures of general purpose stored programme computers
- G06F2015/765—Cache
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Computer Hardware Design (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Memory System Of A Hierarchy Structure (AREA)
Abstract
The application discloses a data processing method, a data processing device, an operation chip and a storage medium. The method is applied to a current operation chip for processing data transmitted by a plurality of upstream operation chips, and comprises the following steps: acquiring the data occupation amount of a cache region; judging whether the data occupation amount reaches a flow control threshold value; if so, acquiring the priority among the upstream operation chips, and sending flow control information to the upstream operation chips according to the sequence of the priority so as to control the upstream operation chips to stop transmitting the data to be processed; performing data processing on data to be processed in the cache region; otherwise, receiving the data to be processed transmitted by the upstream operation chip and writing the data to the cache region; and carrying out data processing on the data to be processed in the cache region. The method relatively ensures that the cloud computing platform realizes the stability of the data acceleration processing process through a plurality of operation chips. In addition, the application also provides a data processing device, an operation chip and a storage medium, and the beneficial effects are as described above.
Description
Technical Field
The present application relates to the field of data processing, and in particular, to a data processing method, apparatus, computing chip and storage medium.
Background
The current cloud computing platform is often formed by a plurality of computing chips, the cloud computing platform uses the computing chip as an acceleration unit, i.e. an acceleration card, in the cloud computing platform, and the plurality of computing chips are connected to a switch through a network interface, such as a network interface of a MAC (multiple access Control) protocol, to form an acceleration resource pool together.
When data to be processed reaches a cloud computing platform, the data to be processed often sequentially passes through a plurality of operation chips in an acceleration resource pool based on a network port transmission mode, when each operation chip in the acceleration resource pool receives the data to be processed, corresponding data processing is carried out on the data through established processing logic of the operation chip, the processed data are further transmitted to a next operation chip to be further processed, the data to be processed are sequentially processed through the operation chips in the acceleration resource pool to obtain data processing results, and an acceleration effect on the data processing process is achieved through a pipeline processing mode among the operation chips. Since the data to be processed need to be sequentially processed by the plurality of operation chips in the acceleration resource pool after reaching the cloud computing platform, further, as shown in fig. 1, there is a current scenario in which the plurality of operation chips provide data to one operation chip for the operation chip to process, that is, there is a current scenario in which the operation chip needs to receive and process data transmitted by a plurality of upstream operation chips, and in this scenario, the reliability of data transmission between the operation chips is ensured, which is an important factor for ensuring the stability of the cloud computing platform when performing data acceleration processing.
Therefore, the data processing method is provided to relatively ensure the reliability of data transmission among the operation chips, and further ensure the stability of the cloud computing platform in the data acceleration processing process through a plurality of operation chips, which is a problem to be solved by technical personnel in the field.
Disclosure of Invention
The application aims to provide a data processing method, a data processing device, an operation chip and a storage medium, so that the reliability of data transmission between the operation chips is relatively ensured, and the stability of a cloud computing platform in the data acceleration processing process through a plurality of operation chips is further ensured.
In order to solve the above technical problem, the present application provides a data processing method applied to a current operation chip for performing data processing on data transmitted by a plurality of upstream operation chips, including:
acquiring the data occupation amount of a cache region;
judging whether the data occupation amount reaches a flow control threshold value;
if so, acquiring the priority among the upstream operation chips, and sending flow control information to the upstream operation chips according to the sequence of the priority so as to control the upstream operation chips to stop transmitting the data to be processed;
performing data processing on data to be processed in the cache region;
otherwise, receiving the data to be processed transmitted by the upstream operation chip and writing the data to the cache region;
and carrying out data processing on the data to be processed in the cache region.
Preferably, when the flow control information is sent to each upstream operation chip in the order of priority, after performing data processing on the data to be processed in the buffer, the method further includes:
monitoring the data occupation amount of the cache region;
and when the data occupation amount of the cache region is smaller than the flow control threshold value, stopping sending the flow control information, and sending flow passing information to the upstream operation chip to control the upstream operation chip to continuously transmit the data to be processed.
Preferably, when there is a downstream operation chip which is adjacent to the current operation chip and performs data processing on the result data generated by the current operation chip, after performing data processing on the data to be processed in the buffer, the method further includes:
transmitting result data generated by data processing on the data to be processed to a downstream operation chip;
and monitoring messages of the downstream operation chip, and stopping transmitting result data to the downstream operation chip when receiving flow control information transmitted by the downstream operation chip.
Preferably, after stopping transmitting the result data to the downstream operation chip, the method further comprises:
and monitoring messages of the downstream operation chip, and continuously transmitting result data to the downstream operation chip when receiving the flow passing information transmitted by the downstream operation chip.
Preferably, the current operation chip includes an FPGA chip.
In addition, the present application further provides a data processing apparatus, which is applied to a current operation chip for performing data processing on data transmitted by a plurality of upstream operation chips, and includes:
the occupancy acquiring module is used for acquiring the data occupancy of the cache region;
the threshold judging module is used for judging whether the data occupation amount reaches a flow control threshold, if so, the control information sending module and the data processing module are called, and otherwise, the receiving module and the data processing module are called;
the control information sending module is used for acquiring the priority among the upstream operation chips and sending flow control information to the upstream operation chips according to the priority order so as to control the upstream operation chips to stop transmitting the data to be processed;
the receiving module is used for receiving the data to be processed transmitted by the upstream operation chip and writing the data into the cache region;
and the data processing module is used for processing the data to be processed in the cache region.
Preferably, the apparatus further comprises:
the occupancy monitoring module is used for monitoring the data occupancy of the cache region;
and the traffic information sending module is used for stopping sending the flow control information when the data occupation amount of the cache region is smaller than the flow control threshold value, and sending flow traffic information to the upstream operation chip so as to control the upstream operation chip to continuously transmit the data to be processed.
Preferably, when there is a downstream operation chip which is adjacent to the current operation chip and performs data processing on the result data generated by the current operation chip, the apparatus further includes:
the result transmission module is used for transmitting result data generated by data processing on the data to be processed to a downstream operation chip;
and the monitoring module is used for monitoring the message of the downstream operation chip and stopping transmitting the result data to the downstream operation chip when receiving the flow control information transmitted by the downstream operation chip.
In addition, this application still provides an operation chip for carry out data processing to the data that upstream operation chip came in, include:
a memory for storing a computer program;
a processor for implementing the steps of the data processing method as described above when executing the computer program.
Furthermore, the present application also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the steps of the data processing method as described above.
The data processing method is applied to a current operation chip for processing data transmitted by a plurality of upstream operation chips, the current operation chip judges whether the data occupation amount reaches a flow control threshold value or not by acquiring the data occupation amount of a cache region, if the data occupation amount reaches the flow control threshold value, the priority among the upstream operation chips is acquired, flow control information is transmitted to the upstream operation chips according to the priority order to control the upstream operation chips to stop transmitting the data to be processed to the current operation chip, and the current operation chip processes the data to be processed in the cache region on the basis; and if the data occupation amount does not reach the flow control threshold value, the current operation chip receives the data to be processed transmitted by the upstream operation chip and writes the data to be processed into the cache region, so that the data to be processed in the cache region is processed. The method judges whether the data occupation amount of the cache region reaches the flow control threshold value or not to reliably receive the data to be processed transmitted by the upstream operation chip, therefore, when the data occupation amount of the cache region reaches the flow control threshold value, the priority among the upstream operation chips is obtained, the flow control information is sent to the upstream operation chips according to the priority order, therefore, the upstream operation chip is stopped from continuously transmitting the data to be processed to the current operation chip, the data to be processed transmitted by the upstream operation chip is received only when the data occupation amount of the buffer area does not reach the flow control threshold value, the condition that the transmitted data to be processed is lost or damaged due to the overflow of the data of the buffer area of the current operation chip is avoided, and the reliability of data transmission among the operation chips is relatively ensured, and the stability of the cloud computing platform in the data acceleration processing process through a plurality of operation chips is further ensured. In addition, the application also provides a data processing device, an operation chip and a storage medium, and the beneficial effects are as described above.
Drawings
In order to more clearly illustrate the embodiments of the present application, the drawings needed for the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.
FIG. 1 is a schematic diagram of a data flow of an arithmetic chip in a cloud computing platform;
FIG. 2 is a flow chart of a data processing method disclosed in an embodiment of the present application;
FIG. 3 is a flow chart of a specific data processing method disclosed in an embodiment of the present application;
fig. 4 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without any creative effort belong to the protection scope of the present application.
When data to be processed reaches a cloud computing platform, the data to be processed often sequentially passes through a plurality of operation chips in an acceleration resource pool based on a network port transmission mode, when each operation chip in the acceleration resource pool receives the data to be processed, corresponding data processing is carried out on the data through established processing logic of the operation chip, the processed data are further transmitted to a next operation chip to be further processed, the data to be processed are sequentially processed through the operation chips in the acceleration resource pool to obtain data processing results, and an acceleration effect on the data processing process is achieved through a pipeline processing mode among the operation chips. After the data to be processed reaches the cloud computing platform, the data needs to be sequentially processed by a plurality of operation chips in the acceleration resource pool, so that the reliability of data transmission among the operation chips is ensured, and the data processing method is an important factor for ensuring the stability of the cloud computing platform in data acceleration processing.
Therefore, the core of the application is to provide a data processing method to relatively ensure the reliability of data transmission among the operation chips, and further ensure the stability of the cloud computing platform in realizing the data acceleration processing process through a plurality of operation chips.
In order that those skilled in the art will better understand the disclosure, the following detailed description will be given with reference to the accompanying drawings.
Referring to fig. 2, an embodiment of the present application discloses a data processing method applied to a current computing chip for performing data processing on data transmitted by a plurality of upstream computing chips, including:
step S10: and acquiring the data occupation amount of the cache region.
In this embodiment, the operation chip that transmits data to be processed to the current operation chip is an upstream operation chip of the current operation chip, and the operation chip that receives data to be processed transmitted by the current operation chip is a downstream operation chip. In practical application scenarios, the upstream operation chip and the current operation chip are relative, and should be determined according to the actual data flow direction between the operation chips. In addition, in this embodiment, the data path between the computing chips in the acceleration resource pool is dynamically adjustable, and the direction of flow control is also dynamically adjustable, so that the flexibility of the data co-processing process between the computing chips can be relatively ensured.
It should be noted that the buffer area in this step refers to a storage space for buffering the data to be processed and intermediate data generated in the data processing process of the data to be processed in the current operation chip. The data occupation amount refers to a data amount occupation degree of existing data in the storage area in the whole storage space.
The current operation chip in this embodiment has a plurality of physical network interfaces, and the upstream operation chip accesses the current operation chip through the physical network interface of the current operation chip and transmits data to be processed to the current operation chip based on the physical network interface.
In addition, in this embodiment, the buffer of the current computing chip may be specifically constructed based on a multi-channel sharing mechanism, that is, the buffer has multiple parallel communication channels, so as to increase the bandwidth for data transmission, and further, the data read-write efficiency of the buffer can be relatively ensured.
In addition, the manner of acquiring the data occupation amount of the cache region may specifically be to acquire a rated storage space size of the cache region and an occupied space size in the storage region, calculate a proportion value of the occupied space size in the rated storage space size, and set the proportion value as the data occupation amount.
Step S11: and judging whether the data occupancy reaches a flow control threshold, if so, executing the step S12 and the step S13, and otherwise, executing the step S14 and the step S15.
Step S12: and acquiring the priority among the upstream operation chips, and sending flow control information to the upstream operation chips according to the priority order so as to control the upstream operation chips to stop transmitting the data to be processed.
It should be noted that, after the current operation chip obtains the data occupancy of the buffer area, it is further determined whether the data occupancy reaches the flow control threshold, and if the data occupancy reaches the flow control threshold, it is determined that the data in the current buffer area is saturated, and in this case, it may happen that the data is continuously stored in the buffer area, and the data in the buffer area overflows.
In the technical scenario of this embodiment, there are a plurality of upstream operation chips that transmit data to be processed to the same current operation chip, that is, the number of the upstream operation chips that transmit data to be processed to the current operation chip is greater than 1, so this embodiment presets priorities among the plurality of upstream operation chips, and aims to send flow control information to each upstream operation chip according to the order of the priorities in the subsequent steps, so as to preferentially control the upstream operation chip with relatively low importance to continuously transmit data to be processed, thereby ensuring the reliability of processing the data to be processed transmitted by the upstream operation chip with relatively high importance.
In addition, the flow control threshold in this embodiment should be determined according to the actual data read-write performance of the buffer area and the relationship between the data load and the stability, and is not specifically limited herein.
Step S13: and carrying out data processing on the data to be processed in the cache region.
After the flow control information is sent to the upstream operation chip to control the upstream operation chip to stop transmitting the data to be processed, the step further performs data processing on the data to be processed in the cache region.
Step S14: and receiving the data to be processed transmitted by the upstream operation chip and writing the data to be processed into the cache region.
Step S15: and carrying out data processing on the data to be processed in the cache region.
And when the data occupation amount is judged not to reach the flow control threshold, the cache region still can continuously and reliably store the data to be processed transmitted by the upstream operation chip, so that in this case, the data to be processed transmitted by the upstream operation chip is received, written into the cache region, and the data to be processed in the cache region is processed.
The data processing method is applied to a current operation chip for processing data transmitted by a plurality of upstream operation chips, the current operation chip judges whether the data occupation amount reaches a flow control threshold value or not by acquiring the data occupation amount of a cache region, if the data occupation amount reaches the flow control threshold value, the priority among the upstream operation chips is acquired, flow control information is transmitted to the upstream operation chips according to the priority order to control the upstream operation chips to stop transmitting the data to be processed to the current operation chip, and the current operation chip processes the data to be processed in the cache region on the basis; and if the data occupation amount does not reach the flow control threshold value, the current operation chip receives the data to be processed transmitted by the upstream operation chip and writes the data to be processed into the cache region, so that the data to be processed in the cache region is processed. The method judges whether the data occupation amount of the cache region reaches the flow control threshold value or not to reliably receive the data to be processed transmitted by the upstream operation chip, therefore, when the data occupation amount of the cache region reaches the flow control threshold value, the priority among the upstream operation chips is obtained, the flow control information is sent to the upstream operation chips according to the priority order, therefore, the upstream operation chip is stopped from continuously transmitting the data to be processed to the current operation chip, the data to be processed transmitted by the upstream operation chip is received only when the data occupation amount of the buffer area does not reach the flow control threshold value, the condition that the transmitted data to be processed is lost or damaged due to the overflow of the data of the buffer area of the current operation chip is avoided, and the reliability of data transmission among the operation chips is relatively ensured, and the stability of the cloud computing platform in the data acceleration processing process through a plurality of operation chips is further ensured.
Referring to fig. 3, an embodiment of the present application discloses a data processing method applied to a current computing chip for performing data processing on data transmitted by a plurality of upstream computing chips, including:
step S20: and acquiring the data occupation amount of the cache region.
Step S21: and judging whether the data occupancy reaches a flow control threshold, if so, executing the steps S22 to S25, and otherwise, executing the steps S26 and S27.
Step S22: and acquiring the priority among the upstream operation chips, and sending flow control information to the upstream operation chips according to the priority order so as to control the upstream operation chips to stop transmitting the data to be processed.
Step S23: and carrying out data processing on the data to be processed in the cache region.
Step S24: and monitoring the data occupation amount of the buffer area.
Step S25: and when the data occupation amount of the cache region is smaller than the flow control threshold value, stopping sending the flow control information, and sending flow passing information to the upstream operation chip to control the upstream operation chip to continuously transmit the data to be processed.
Step S26: and receiving the data to be processed transmitted by the upstream operation chip and writing the data to be processed into the cache region.
Step S27: and carrying out data processing on the data to be processed in the cache region.
It should be noted that, in this embodiment, after the flow control information is sent to the upstream operation chip to control the upstream operation chip to stop transmitting the to-be-processed data, the data occupation amount of the buffer area is further continuously monitored, and when the data occupation amount of the buffer area is smaller than the flow control threshold, the flow control information is stopped being sent, and the flow passing information is sent to the upstream operation chip, so as to control the upstream operation chip to continuously transmit the to-be-processed data, thereby further ensuring that the to-be-processed data newly generated by the upstream operation chip can be normally transmitted to the current operation chip, thereby ensuring that the current operation chip can normally process the to-be-processed data newly generated by the upstream operation chip, and further ensuring the reliability of data transmission between the operation chips.
On the basis of the foregoing embodiment, as a preferred implementation manner, when there is a downstream operation chip which is adjacent to the current operation chip and performs data processing on result data generated by the current operation chip, after performing data processing on data to be processed in the buffer, the method further includes:
transmitting result data generated by data processing on the data to be processed to a downstream operation chip;
and monitoring messages of the downstream operation chip, and stopping transmitting result data to the downstream operation chip when receiving flow control information transmitted by the downstream operation chip.
In the scenario of the present embodiment, there is a downstream computing chip that is adjacent to the current computing chip and performs data processing on the result data output by the current computing chip as the data to be processed, in the scene, the result data generated by data processing of the data to be processed is transmitted to the downstream operation chip, and the message monitoring is carried out on the downstream operation chip, and when the flow control information transmitted by the downstream operation chip is received, the transmission of the result data to the downstream operation chip is stopped, so that the problem that the data occupation amount of a cache region in the downstream operation chip is large is further avoided, and the result data transmitted by the current operation chip to the downstream operation chip is lost or damaged, therefore, the reliability of the data transmission process of the current operation chip to the downstream operation chip is further ensured, and the reliability of data transmission between the operation chips is ensured.
Further, as a preferred embodiment, after stopping transmitting the result data to the downstream operation chip, the method further includes:
and monitoring messages of the downstream operation chip, and continuously transmitting result data to the downstream operation chip when receiving the flow passing information transmitted by the downstream operation chip.
In the embodiment, after the current operation chip stops transmitting the result data to the downstream operation chip, the message monitoring is further performed on the downstream operation chip, and then when the flow traffic information transmitted by the downstream operation chip is received, the result data is continuously transmitted to the downstream operation chip, so that the result data in the current operation chip can be continuously transmitted to the downstream operation chip after the data occupancy amount in the cache region of the downstream operation chip is recovered to a normal level, the reliability of the data transmission process performed by the current operation chip to the downstream operation chip is further ensured, and the reliability of data transmission between the operation chips is ensured.
On the basis of some of the above-mentioned embodiments, as a preferred embodiment, the current operation chip includes an FPGA chip.
It should be noted that, because an FPGA (Field Programmable Gate Array) chip has flexibility and high-speed operational capability, and has higher data parallel processing performance, when an FPGA cloud computing platform composed of a plurality of FPGA chips performs data processing, pipeline processing of data can be more efficiently realized between the FPGA chips, and overall efficiency of realizing a data accelerated processing process through the plurality of FPGA operational chips is further ensured.
Referring to fig. 4, an embodiment of the present application provides a data processing apparatus applied to a current computing chip for performing data processing on data incoming from a plurality of upstream computing chips, including:
the occupancy acquiring module 10 is used for acquiring the data occupancy of the cache region;
and the threshold value judging module 11 is used for judging whether the data occupation amount reaches a flow control threshold value, if so, calling the control information sending module 12 and the data processing module 14, and otherwise, calling the receiving module 13 and the data processing module 14.
And the control information sending module 12 is configured to obtain priorities among the upstream operation chips, and send flow control information to the upstream operation chips according to the order of the priorities, so as to control the upstream operation chips to stop transmitting data to be processed.
The receiving module 13 is configured to receive data to be processed and sent from an upstream computing chip, and write the data to the buffer.
And the data processing module 14 is used for performing data processing on the data to be processed in the buffer area.
The data processing device is applied to a current operation chip which performs data processing on data transmitted by a plurality of upstream operation chips, the current operation chip judges whether the data occupation amount reaches a flow control threshold value or not by acquiring the data occupation amount of a cache region, if the data occupation amount reaches the flow control threshold value, the sending of flow control information is stopped so as to control the upstream operation chips to stop transmitting data to be processed to the current operation chip, and the current operation chip performs data processing on the data to be processed in the cache region on the basis; and if the data occupation amount does not reach the flow control threshold value, the current operation chip receives the data to be processed transmitted by the upstream operation chip and writes the data to be processed into the cache region, so that the data to be processed in the cache region is processed. The device judges whether the data occupation amount of the cache region reaches the flow control threshold value or not, and can reliably receive the data to be processed transmitted by the upstream operation chip, so that the sending of flow control information is stopped when the data occupation amount of the cache region reaches the flow control threshold value, the upstream operation chip is stopped from continuously transmitting the data to be processed to the current operation chip, the data to be processed transmitted by the upstream operation chip is received only when the data occupation amount of the cache region does not reach the flow control threshold value, the condition that the transmitted data to be processed is lost or damaged due to the overflow of the data of the cache region of the current operation chip is avoided, the reliability of data transmission between the operation chips is relatively ensured, and the stability of the cloud computing platform in the data acceleration processing process through a plurality of operation chips is further ensured.
Further, as a preferred embodiment, the apparatus further comprises:
the occupancy monitoring module is used for monitoring the data occupancy of the cache region;
and the traffic information sending module is used for stopping sending the flow control information when the data occupation amount of the cache region is smaller than the flow control threshold value, and sending flow traffic information to the upstream operation chip so as to control the upstream operation chip to continuously transmit the data to be processed.
In addition, as a preferred embodiment, when there is a downstream operation chip which is adjacent to the current operation chip and performs data processing on result data generated by the current operation chip, the apparatus further includes:
the result transmission module is used for transmitting result data generated by data processing on the data to be processed to a downstream operation chip;
and the monitoring module is used for monitoring the message of the downstream operation chip and stopping transmitting the result data to the downstream operation chip when receiving the flow control information transmitted by the downstream operation chip.
In addition, an embodiment of the present application further provides an operation chip, configured to perform data processing on data transmitted by an upstream operation chip, including:
a memory for storing a computer program;
a processor for implementing the steps of the data processing method as described above when executing the computer program.
The arithmetic chip provided by the application is used for carrying out data processing on data transmitted by an upstream arithmetic chip, the arithmetic chip judges whether the data occupation amount reaches a flow control threshold value or not by acquiring the data occupation amount of a cache region, if the data occupation amount reaches the flow control threshold value, the flow control information is stopped being sent so as to control the upstream arithmetic chip to stop transmitting data to be processed to the arithmetic chip, and the arithmetic chip carries out data processing on the data to be processed in the cache region on the basis; and if the data occupation amount does not reach the flow control threshold value, the arithmetic chip receives the data to be processed transmitted by the upstream arithmetic chip and writes the data to be processed into the cache region, so that the data to be processed in the cache region is processed. The operation chip judges whether to reliably receive the data to be processed transmitted by the upstream operation chip by judging whether the data occupation amount of the cache region reaches a flow control threshold value, stops transmitting flow control information when the data occupation amount of the cache region reaches the flow control threshold value, stops the upstream operation chip from continuously transmitting the data to be processed to the current operation chip, and receives the data to be processed transmitted by the upstream operation chip only when the data occupation amount of the cache region does not reach the flow control threshold value, so that the condition that the transmitted data to be processed is lost or damaged due to overflow of the data of the cache region of the current operation chip is avoided, the reliability of data transmission among the operation chips is relatively ensured, and the stability of a cloud computing platform for realizing a data acceleration processing process through a plurality of operation chips is further ensured.
In addition, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the data processing method as described above.
The computer-readable storage medium provided by the application is applied to a current operation chip for carrying out data processing on data transmitted by a plurality of upstream operation chips, the current operation chip judges whether the data occupation amount reaches a flow control threshold value or not by acquiring the data occupation amount of a cache region, if the data occupation amount reaches the flow control threshold value, the sending of flow control information is stopped so as to control the upstream operation chips to stop transmitting data to be processed to the current operation chip, and the current operation chip carries out data processing on the data to be processed in the cache region on the basis; and if the data occupation amount does not reach the flow control threshold value, the current operation chip receives the data to be processed transmitted by the upstream operation chip and writes the data to be processed into the cache region, so that the data to be processed in the cache region is processed. The computer readable storage medium judges whether to reliably receive data to be processed transmitted by an upstream operation chip by judging whether the data occupation amount of a cache region reaches a flow control threshold value, stops transmitting flow control information when the data occupation amount of the cache region reaches the flow control threshold value, stops the upstream operation chip from continuously transmitting the data to be processed to a current operation chip, and receives the data to be processed transmitted by the upstream operation chip only when the data occupation amount of the cache region does not reach the flow control threshold value, so that the condition that the transmitted data to be processed is lost or damaged due to overflow of the data of the cache region of the current operation chip is avoided, the reliability of data transmission among operation chips is relatively ensured, and the stability of a cloud computing platform for realizing a data acceleration processing process through a plurality of operation chips is further ensured.
The data processing method, the data processing apparatus, the computing chip and the storage medium provided by the present application are described in detail above. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A data processing method is applied to a current operation chip which performs data processing on data transmitted by a plurality of upstream operation chips, and comprises the following steps:
acquiring the data occupation amount of a cache region;
judging whether the data occupation amount reaches a flow control threshold value;
if so, acquiring the priority among the upstream operation chips, and sending flow control information to the upstream operation chips according to the sequence of the priority so as to control the upstream operation chips to stop transmitting the data to be processed;
performing data processing on the data to be processed in the cache region;
otherwise, receiving the data to be processed transmitted by the upstream operation chip and writing the data to be processed into the cache region;
and performing data processing on the data to be processed in the cache region.
2. The data processing method according to claim 1, wherein when sending flow control information to each of the upstream arithmetic chips in the order of the priority, after the data processing is performed on the data to be processed in the buffer, the method further comprises:
monitoring the data occupation amount of the cache region;
and when the data occupation amount of the cache region is smaller than the flow control threshold value, stopping sending the flow control information, and sending flow passing information to the upstream operation chip to control the upstream operation chip to continuously transmit the data to be processed.
3. The data processing method according to claim 2, wherein when there is a downstream operation chip which is adjacent to the current operation chip and performs data processing on result data generated by the current operation chip, after the data processing on the data to be processed in the buffer, the method further comprises:
transmitting result data generated by data processing on the data to be processed to the downstream operation chip;
and monitoring the message of the downstream operation chip, and stopping transmitting the result data to the downstream operation chip when the flow control information transmitted by the downstream operation chip is received.
4. The data processing method of claim 3, wherein after the stopping of the transmission of the result data to the downstream compute chip, the method further comprises:
and monitoring the message of the downstream operation chip, and continuously transmitting the result data to the downstream operation chip when receiving the flow passing information transmitted by the downstream operation chip.
5. The data processing method of any one of claims 1 to 4, wherein the current operational chip comprises an FPGA chip.
6. A data processing device, applied to a current operation chip for performing data processing on data incoming from a plurality of upstream operation chips, comprises:
the occupancy acquiring module is used for acquiring the data occupancy of the cache region;
the threshold value judging module is used for judging whether the data occupation amount reaches a flow control threshold value, if so, the control information sending module and the data processing module are called, and otherwise, the receiving module and the data processing module are called;
the control information sending module is used for acquiring the priority among the upstream operation chips and sending flow control information to the upstream operation chips according to the priority order so as to control the upstream operation chips to stop transmitting data to be processed;
the receiving module is used for receiving the data to be processed transmitted by the upstream operation chip and writing the data to be processed into the cache region;
and the data processing module is used for performing data processing on the data to be processed in the cache region.
7. The data processing apparatus of claim 6, wherein the apparatus further comprises:
the occupancy monitoring module is used for monitoring the data occupancy of the cache region;
and the traffic information sending module is used for stopping sending the flow control information when the data occupation amount of the cache region is smaller than the flow control threshold value, and sending flow traffic information to the upstream operation chip so as to control the upstream operation chip to continuously transmit the data to be processed.
8. The data processing apparatus according to claim 7, wherein when there is a downstream operation chip which is adjacent to the current operation chip and performs data processing on result data generated by the current operation chip, the apparatus further comprises:
the result transmission module is used for transmitting result data generated by data processing on the data to be processed to the downstream operation chip;
and the monitoring module is used for monitoring the message of the downstream operation chip and stopping transmitting the result data to the downstream operation chip when the flow control information transmitted by the downstream operation chip is received.
9. An arithmetic chip for data processing of data incoming from an upstream arithmetic chip, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the data processing method according to any one of claims 1 to 5 when executing the computer program.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the data processing method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010529663.8A CN111756650A (en) | 2020-06-11 | 2020-06-11 | Data processing method and device, operation chip and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010529663.8A CN111756650A (en) | 2020-06-11 | 2020-06-11 | Data processing method and device, operation chip and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111756650A true CN111756650A (en) | 2020-10-09 |
Family
ID=72675090
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010529663.8A Pending CN111756650A (en) | 2020-06-11 | 2020-06-11 | Data processing method and device, operation chip and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111756650A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1988496A (en) * | 2005-12-21 | 2007-06-27 | 华为技术有限公司 | Method for regulating optic fiber path data flow speed rate |
| CN102096648A (en) * | 2010-12-09 | 2011-06-15 | 深圳中兴力维技术有限公司 | System and method for realizing multipath burst data business caching based on FPGA (Field Programmable Gate Array) |
| CN105183431A (en) * | 2015-08-05 | 2015-12-23 | 瑞斯康达科技发展股份有限公司 | Method and apparatus for controlling CPU utilization ratio |
| CN108574870A (en) * | 2018-04-13 | 2018-09-25 | 郑州云海信息技术有限公司 | A kind of display methods, device and the equipment in multi channel signals source |
| CN109582442A (en) * | 2018-11-30 | 2019-04-05 | 北京微播视界科技有限公司 | Processing method, electronic equipment and the computer storage medium of memory spilling OOM |
| CN109818879A (en) * | 2017-11-22 | 2019-05-28 | 华为技术有限公司 | Flow control methods and device |
-
2020
- 2020-06-11 CN CN202010529663.8A patent/CN111756650A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1988496A (en) * | 2005-12-21 | 2007-06-27 | 华为技术有限公司 | Method for regulating optic fiber path data flow speed rate |
| CN102096648A (en) * | 2010-12-09 | 2011-06-15 | 深圳中兴力维技术有限公司 | System and method for realizing multipath burst data business caching based on FPGA (Field Programmable Gate Array) |
| CN105183431A (en) * | 2015-08-05 | 2015-12-23 | 瑞斯康达科技发展股份有限公司 | Method and apparatus for controlling CPU utilization ratio |
| CN109818879A (en) * | 2017-11-22 | 2019-05-28 | 华为技术有限公司 | Flow control methods and device |
| CN108574870A (en) * | 2018-04-13 | 2018-09-25 | 郑州云海信息技术有限公司 | A kind of display methods, device and the equipment in multi channel signals source |
| CN109582442A (en) * | 2018-11-30 | 2019-04-05 | 北京微播视界科技有限公司 | Processing method, electronic equipment and the computer storage medium of memory spilling OOM |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106664290B (en) | data transmission method and device of photoelectric hybrid network | |
| US9215174B2 (en) | Oversubscription buffer management | |
| CN103442391A (en) | Data transmission method and system as well as user terminal | |
| CN103493449A (en) | Effective circuits in packet-switched networks | |
| CN101616083A (en) | A kind of message forwarding method and device | |
| US20160196073A1 (en) | Memory Module Access Method and Apparatus | |
| US11671678B2 (en) | Method and device, equipment, and storage medium for data processing | |
| CN102611630B (en) | A kind of message acceptance control method and system | |
| RU2641250C2 (en) | Device and method of queue management | |
| CN113254375A (en) | Data transmission method and device, electronic equipment and storage medium | |
| CN116233018A (en) | Message processing method and device, electronic equipment and storage medium | |
| CN110557432B (en) | Cache pool balance optimization method, system, terminal and storage medium | |
| CN109120539B (en) | Method and device for realizing data transmission processing | |
| CN110245013B (en) | Internet of Things computing resource management method and device | |
| CN111756650A (en) | Data processing method and device, operation chip and storage medium | |
| CN115967687A (en) | Method for fast forwarding message, network device, storage medium and computer program product | |
| CN113157465B (en) | Message sending method and device based on pointer linked list | |
| CN111131083B (en) | Method, device and equipment for data transmission between nodes and computer readable storage medium | |
| JP2011091711A (en) | Node, method for distributing transmission frame, and program | |
| CN117499351A (en) | Message forwarding device and method, communication chip and network equipment | |
| CN114500544A (en) | Method, system, equipment and medium for load balancing among nodes | |
| WO2022174444A1 (en) | Data stream transmission method and apparatus, and network device | |
| CN115878309A (en) | Resource allocation method, device, processing core, equipment and computer readable medium | |
| CN113453285A (en) | Resource adjusting method, device and storage medium | |
| EP4135225A1 (en) | Service flow transmission method and apparatus, and device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201009 |