CN109032704B - Data processing method and device - Google Patents
Data processing method and device Download PDFInfo
- Publication number
- CN109032704B CN109032704B CN201710439303.7A CN201710439303A CN109032704B CN 109032704 B CN109032704 B CN 109032704B CN 201710439303 A CN201710439303 A CN 201710439303A CN 109032704 B CN109032704 B CN 109032704B
- Authority
- CN
- China
- Prior art keywords
- data processing
- data
- processing unit
- unit
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Programmable Controllers (AREA)
- Stored Programmes (AREA)
Abstract
The embodiment of the invention discloses a data processing method, which comprises the following steps: acquiring data to be processed; determining N data processing units for executing data processing operation according to the data processing requirement of the data to be processed, and determining the execution sequence of the N data processing units when executing the data processing operation; each data processing unit is a functional unit for performing a type of data processing operation, N is an integer greater than 1; and performing data processing on the data to be processed according to the execution sequence. The embodiment of the invention also discloses data processing equipment.
Description
Technical Field
The present invention relates to the field of data processing, and in particular, to a method and an apparatus for data processing.
Background
With the development of networks, data processing that the network processing chip needs to support becomes more and more complex, and after the network processing chip is manufactured, the network processing chip cannot support new data processing.
With the increasingly complex operation of the network processing chip on data processing, the flow of data processing is increased continuously, and the existing network processing chip is difficult to support the upgrading operation of data processing when processing data through the embedded programmable module. All data processing possibilities cannot be considered in the design process of the network processing chip under the influence of uncertainty and flexibility of data processing, and when the embedded programmable module realizes expandability and programmability of the network processing chip, the expandability of the network processing chip is poor due to the fixed hardware structure of the embedded programmable module; the programmability of the network processing chip is also poor in the case that the internal resources of the network processing chip are limited.
Disclosure of Invention
In order to solve the foregoing technical problems, embodiments of the present invention provide a data processing method and device, which can set an execution sequence of a data processing unit according to a data processing requirement, thereby implementing different data processing operations, and thus, improve extensibility and programmability of data processing.
The technical scheme of the invention is realized as follows:
in a first aspect, an embodiment of the present invention provides a data processing method, including: acquiring data to be processed; determining N data processing units executing data processing operation and the execution sequence of the N data processing units when executing the data processing operation according to the data processing requirement of the data to be processed; each data processing unit is a functional unit for performing a type of data processing operation; n is an integer greater than 1; and performing data processing on the data to be processed according to the execution sequence.
In the foregoing solution, the performing data processing on the data to be processed according to the execution sequence includes: for the ith data processing unit of the N data processing units, when i is 1, 2, 3, …, N-1, executing the following steps: sending the data to be processed of the ith data processing unit to the ith data processing unit for data processing; receiving processed data from the ith data processing unit; determining the (i + 1) th data processing unit according to the execution sequence; and sending the processed data to the (i + 1) th data processing unit for data processing.
In the foregoing solution, the performing data processing on the data to be processed according to the execution sequence includes: for the ith data processing unit of the N data processing units, when i is 1, 2, 3, …, N-1, executing the following steps: sending the data to be processed of the ith data processing unit to the ith data processing unit for data processing; determining the (i + 1) th data processing unit according to the execution sequence; and controlling the (i + 1) th data processing unit to perform data processing on the processed data of the (i) th data processing unit.
In the foregoing solution, before the controlling the i +1 th data processing unit to perform data processing on the processed data of the i-th data processing unit, the method further includes: determining that K data processing units exist between the ith data processing unit and the (i + 1) th data processing unit according to the execution sequence, wherein K is an integer greater than 0; and configuring each data processing unit in the K data processing units into a transparent transmission state, wherein the transparent transmission state is used for indicating each data processing unit in the K data processing units to transmit input data.
In the above scheme, the data processing operation includes: insert, delete, modify, operate on, and mark.
In a second aspect, an embodiment of the present invention further provides a data processing apparatus, where the apparatus includes: a processor and N data processing units; wherein the processor is configured to perform the steps of: acquiring data to be processed; determining N data processing units for executing data processing operation according to the data processing requirement of the data to be processed, and determining the execution sequence of the N data processing units when executing the data processing operation; each data processing unit is a functional unit for performing a type of data processing operation, N is an integer greater than 1; and performing data processing on the data to be processed according to the execution sequence of the N data processing units when executing data processing operation.
In the foregoing solution, the processor is specifically configured to execute the following steps: for the ith data processing unit of the N data processing units, when i is 1, 2, 3, …, N-1, executing the following steps: sending the data to be processed of the ith data processing unit to the ith data processing unit for data processing; receiving processed data from the ith data processing unit; determining the (i + 1) th data processing unit according to the execution sequence; and sending the processed data to the (i + 1) th data processing unit for data processing.
In the foregoing solution, the processor is specifically configured to execute the following steps: for the ith data processing unit of the N data processing units, when i is 1, 2, 3, …, N-1, executing the following steps: sending the data to be processed of the ith data processing unit to the ith data processing unit for data processing; determining the (i + 1) th data processing unit according to the execution sequence; and controlling the (i + 1) th data processing unit to perform data processing on the processed data of the (i) th data processing unit.
In the foregoing solution, before the step of controlling the i +1 th data processing unit to perform data processing on the processed data of the i-th data processing unit, the processor is further configured to perform the following steps: determining that K data processing units exist between the ith data processing unit and the (i + 1) th data processing unit according to the execution sequence, wherein K is an integer greater than 0; and configuring each data processing unit in the K data processing units into a transparent transmission state, wherein the transparent transmission state is used for indicating each data processing unit in the K data processing units to transmit input data.
In a third aspect, the present invention also provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the method of any one of the above data processing.
The data processing method and the data processing equipment provided by the embodiment of the invention are used for acquiring data to be processed; determining N data processing units executing data processing operation and the execution sequence of the N data processing units when executing the data processing operation according to the data processing requirement of the data to be processed; each data processing unit is a functional unit for performing a type of data processing operation, N being an integer greater than 1; and performing data processing on the data to be processed according to the execution sequence. Compared with the prior art, different data processing operations can be realized through different data processing units, and then the execution sequence of the corresponding data processing unit is configured according to different data processing requirements to complete different data processing flows, so that the expandability and the programmability of data processing are improved.
Drawings
FIG. 1 is a flowchart illustrating a first embodiment of a method for data processing according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a second embodiment of a method for data processing according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram illustrating a first operating mode of a data processing unit according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram illustrating a second operating mode of a data processing unit according to an embodiment of the present invention;
FIG. 5 is a flow chart illustrating data processing operations performed in an embodiment of the present invention;
FIG. 6 is a flowchart illustrating a data processing method according to a third embodiment of the present invention;
FIG. 7 is a schematic structural diagram illustrating a third operating mode of a data processing unit according to an embodiment of the present invention;
fig. 8 is a schematic diagram of the configuration of the data processing main unit X in the embodiment of the present invention;
fig. 9 is a schematic diagram of the configuration of the data processing main unit Y in the embodiment of the present invention;
fig. 10 is a schematic diagram of the configuration of a data processing main unit Z in the embodiment of the present invention;
fig. 11 is a schematic diagram of a hardware structure of a data processing apparatus according to an embodiment of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.
The embodiment of the invention provides a data processing method, which can be applied to a network processing chip, wherein the chip can comprise: n data processing units and controllers; the N data processing units can be connected through a bus and carry out mutual communication, and the controller controls the N data processing units through the bus to carry out data processing on data to be processed according to a certain execution sequence.
In practical implementation, the data processing unit is a functional unit for performing a type of data processing operation, and the functional unit may be specifically implemented by a circuit or by a program code.
For example, the data processing operations performed by the data processing unit may include the following:
1) inserting operation (ACTION _ INSERT), inserting Q (Q is 1, 2, 4, etc., expandable) byte contents at the corresponding source message offset src _ offset, that is, contents with lengths of 8bit, 16bit, 32bit, etc. can be inserted at a single time, having expandability, and moving back the message contents at the corresponding position, and increasing the length of the message (descriptor). It should be noted that the unit of the insert operation and all subsequent data processing operations is bytes. In the embodiment of the present invention, Q is a positive integer, and src _ offset refers to an offset from a data start position in a source message.
2) A first delete operation (ACTION _ REMOVE) that deletes Q (Q ═ 1, 2, 4, etc., extensible) byte contents at the corresponding src _ offset, and advances the subsequent packet by Q bytes; the message (descriptor) length is reduced.
3) And modifying operation (ACTION _ SET), namely modifying the content of Q bytes (Q is 1, 2, 4 and the like, and can be expanded) at the corresponding src _ offset, wherein the content of the rest messages is unchanged, and the length of the messages (descriptors) is unchanged.
4) A replacement operation (ACTION _ MOVE) of replacing the modified Q (Q ═ 1, 2, 4, etc., extensible) byte contents at the src _ offset corresponding to the message with Q (Q ═ 1, 2, 4, etc., extensible) byte contents at the destination message offset dst _ offset; the length of the message (descriptor) is not changed, dst _ offset is the offset from the beginning of the data in the destination message.
5) An ADD operation (ACTION _ ADD) replaces the content of Q (Q is 1, 2, 4, etc., scalable) bytes of data at src _ offset with: (source message content + action _ data), the length of the message (descriptor) is not changed, and the action _ data refers to the added content.
6) A subtract operation (ACTION _ SUB) replaces the content of Q bytes of data at src _ offset (Q is 1, 2, 4, etc., scalable) with: (source message content-action _ data), the length of the message (descriptor) is not changed, and the action _ data refers to the reduced content.
7) And operation (ACTION _ MASK), replacing the data N (1, 2, 4) byte at src _ offset with: (source message content & MASK), the length of the message (descriptor) is not changed, and MASK refers to the content participating in the and operation.
8) Comparing operation (ACTION _ fiter), comparing the message content of the data N (1, 2, 4) bytes at src _ offset with the fiter content, and determining the message operation according to the comparison result, such as: transparent transmission, upload, discard, etc.
9) The first marking operation (ACTION _ CheckSum), takes the corresponding CheckSum calculation mark, for example: TCP checksum calculation flag, UDP checksum calculation flag, IP checksum calculation flag, and the like. Wherein, TCP is an english abbreviation of Transmission Control Protocol (Transmission Control Protocol), UPD is an english abbreviation of User Datagram Protocol (User data Protocol), and IP is an english abbreviation of Internet Protocol (Internet Protocol).
10) And performing a second marking operation (ACTION _ CRC), marking a corresponding Cyclic Redundancy Check (CRC) calculation mark, and supporting various CRC calculation marks according to requirements.
11) An extended function (ACTION _ NON), the added new processing operation can be implemented with the unit having this function.
12) Connecting operation (ACTION _ CONNECT), namely connecting the front and back beat data according to valid data, and eliminating the middle invalid data; the data inserted into the whole beat can also be obtained by expanding the data processing unit executing the operation, and theoretically, the increasing beat and the decreasing beat of the data have no influence.
13) And a second delete operation (ACTION _ DEL) of discarding the entire beat data after the entire beat data is deleted.
In actual implementation, the data processing unit may implement one or more of the data processing operations described above.
The controller may be at least one of a Central Processing Unit (CPU), a microprocessor Unit (MPU), an Integrated Circuit (ASIC), a Digital Signal Processing Device (DSPD), and a Programmable Logic Device (PLD). It is to be understood that the electronic devices for implementing the above controller functions may be other chips, and the embodiments of the present invention are not limited in particular.
Fig. 1 is a schematic flowchart of a first embodiment of a data processing method according to an embodiment of the present invention, as shown in fig. 1, the method includes:
step 101: and acquiring data to be processed.
Step 102: determining N data processing units executing data processing operation and the execution sequence of the N data processing units when executing the data processing operation according to the data processing requirement of the data to be processed; each data processing unit is a functional unit for performing a type of data processing operation, and N is an integer greater than 1.
Illustratively, after obtaining the data processing instruction, N data processing units corresponding to the data to be processed are determined according to the requirement of the data to be processed, and the execution order of the N data processing units when executing the data processing operation is determined, and further, the execution order of the N data processing units is configured according to the determined execution order.
Specifically, a mapping list may be pre-established, where the mapping list is used to indicate the execution order of the N data processing units corresponding to different data processing requirements. After the data to be processed is obtained, the execution sequence of the N data processing units is determined through table lookup according to the requirement of the data to be processed, and then the execution sequence of the N data processing units is configured according to the determined execution sequence.
In actual implementation, the data processing operations may include: insert, delete, modify, operate on, and mark. Each data processing unit is configured to perform a type of data processing operation, which may include one or more data processing operations. It will be appreciated that, in order to ensure resource utilization of the data processing unit, when a class of data processing operations includes a plurality of data processing operations, there should be a high similarity between the processing logic of the plurality of data processing operations. For example, the addition operation and the subtraction operation may be performed by one data processing unit.
Illustratively, providing 4 data processing units of different functions includes: the data processing unit A, the data processing unit B, the data processing unit C and the data processing unit D require inconsistent execution sequence under different application scenes. For example: the execution sequence of the 4 data processing units when executing the data processing operation includes 24 types, which specifically includes: from A to B to C to D, from B to C to D to A, and from C to B to D to A, and so on.
Step 103: and performing data processing on the data to be processed according to the execution sequence.
A first optional implementation manner in this embodiment of the present invention is that each data processing unit is connected to the controller through a bus, and the controller may configure the operation mode of each data processing unit to obtain an execution sequence of the data processing units when performing data processing operations, so that, in order to reduce the configuration workload of the controller, the controller may configure only N data processing units actually used for performing data processing.
At this time, the step 103 may include: for the ith data processing unit of the N data processing units, when i is 1, 2, 3, …, N-1, executing the following steps: sending the data to be processed of the ith data processing unit to the ith data processing unit for data processing; receiving processed data from an ith data processing unit; determining the (i + 1) th data processing unit according to the execution sequence; and sending the processed data to the (i + 1) th data processing unit for data processing.
Illustratively, determining 3 data processing units to perform data processing operations according to processing requirements of data to be processed includes: A. b and D, and the execution sequence of the 3 data processing units when executing the data processing operation is as follows: from B to D to A. According to the execution sequence, the specific steps of performing data processing on the data to be processed may include:
step 1: the controller sends the data to be processed to the unit B;
and 2, step: receiving processed data from unit B;
and step 3: sending the processed data of the unit B to a unit D;
and 4, step 4: receiving processed data from unit D;
and 5: sending the processed data of the unit D to the unit A;
step 6: and outputting the processed data of the unit A.
In a second alternative implementation, the data processing units are connected in sequence according to a preset arrangement order, and any two adjacent data processing units can communicate with each other. Then, in order to reduce the signaling interaction on the bus, data may be controlled to be transferred between the data processing units to perform data processing operations.
At this time, the step 103 may include: for the ith data processing unit of the N data processing units, when i is 1, 2, 3, …, N-1, performing the steps of: sending the data to be processed of the ith data processing unit to the ith data processing unit for data processing; determining the (i + 1) th data processing unit according to the execution sequence; and controlling the (i + 1) th data processing unit to perform data processing on the processed data of the ith data processing unit. In this embodiment, signaling interaction on the bus is reduced by controlling the transmission of data between the data processing units.
Illustratively, connecting the data processing units A, B, C, D in the order from a to B to C to D, determining 3 data processing units to perform data processing operations according to the processing requirements of the data to be processed includes: A. b and C, and the execution sequence of the 3 data processing units when executing the data processing operation is as follows: from A to B to C. According to the execution sequence, the specific steps of performing data processing on the data to be processed may include:
step 1: the controller sends the data to be processed to the unit A;
step 2: determining the unit B as a unit for executing the data processing operation next according to the execution sequence;
and step 3: the control unit B performs data processing on the processed data of the unit A;
and 4, step 4: determining the unit C as a unit for executing the data processing operation next according to the execution sequence;
and 5: the control unit C performs data processing on the processed data of the unit B;
step 6: and outputting the processed data of the unit C.
A third optional implementation manner is that, on the basis of the second optional implementation manner, if there is another data processing unit between two data processing units adjacent to each other in the execution sequence in the N data processing units, the data can still be controlled to be transmitted between the data processing units by configuring the other data processing units in the transparent transmission state.
At this time, the step 103 may include: for the ith data processing unit of the N data processing units, when i is 1, 2, 3, …, N-1, executing the following steps: sending the data to be processed of the ith data processing unit to the ith data processing unit for data processing; determining the (i + 1) th data processing unit according to the execution sequence; determining that K data processing units exist between the ith data processing unit and the (i + 1) th data processing unit according to the execution sequence, wherein K is an integer greater than 0; and configuring each data processing unit in the K data processing units into a transparent transmission state, wherein the transparent transmission state is used for indicating each data processing unit in the K data processing units to transmit input data. In this embodiment, the controller may control data to be transmitted between any two data processing units according to different data processing requirements, thereby reducing signaling interaction on the bus and improving flexibility of execution sequence configuration of the data processing units.
Illustratively, connecting the data processing units A, B, C, D in the order from a to B to C to D, determining 3 data processing units to perform data processing operations according to the processing requirements of the data to be processed includes: A. b and C, and 3 data processing units are executed in the following order when executing the data processing operation: from B to C to A. According to the execution sequence, the specific steps of performing data processing on the data to be processed may include:
step 1: the controller sends the data to be processed to the unit B;
step 2: the control unit C performs data processing on the processed data of the unit B;
and step 3: determining that a unit B exists between the unit C and a next unit A which performs a data processing operation;
and 4, step 4: configuring the unit B into a transparent transmission state;
and 5: the control unit A performs data processing on the processed data of the unit C;
step 6: and outputting the processed data of the unit A.
In the embodiment of the invention, data to be processed is obtained; determining N data processing units executing data processing operation and the execution sequence of the N data processing units when executing the data processing operation according to the data processing requirement of the data to be processed; each data processing unit is a functional unit for performing a type of data processing operation, N is an integer greater than 1; and performing data processing on the data to be processed according to the execution sequence. Compared with the prior art, different data processing operations can be realized through different data processing units, and then the execution sequence of the corresponding data processing unit is configured according to different data processing requirements to complete different data processing flows, so that the expandability and the programmability of data processing are improved.
Second embodiment
Based on the foregoing embodiments, in this embodiment, taking 4 data processing units as an example to perform data processing, the data processing method provided by the embodiment of the present invention is further described.
Assume that the above N data processing units are data processing unit a, data processing unit B, data processing unit C, and data processing unit D.
Fig. 2 is a flowchart illustrating a second embodiment of a data processing method according to an embodiment of the present invention, as shown in fig. 2, the method includes:
step 201: and acquiring data to be processed.
Step 202: and determining 4 data processing units for executing the data processing operation and the execution sequence of the 4 data processing units when executing the data processing operation according to the data processing requirement of the data to be processed.
In practical implementation, the data processing units A, B, C and D have 24 operation modes (i.e. 24 execution sequences) for different application scenarios, and the specific operation modes include: data flow in turn goes through A, B, C, D in mode 1, B, C, D, A in mode 2, C, B, D, A in mode 3, and so on. From the 24 operation modes, an operation mode is selected for data processing of the data to be processed according to the data processing requirement.
Step 203: the execution order of the 4 data processing units is configured according to the determined execution order.
Step 204: and carrying out data processing on the data to be processed.
Exemplarily, fig. 3 is a schematic structural diagram of a first operation mode of a data processing unit in an embodiment of the present invention, and as shown in fig. 3, a data processing unit a301, a data processing unit B302, a data processing unit C303, a data processing unit D304, and an output unit 305 are connected to a controller 306 through a bus, and the controller is configured to configure an execution sequence of the data processing units. Optionally, each data processing unit is connected according to a preset arrangement order (i.e. from a to B to C to D), and each data processing unit may further include a selection subunit and a processing subunit, where the selection subunit is configured to control the respective data processing unit to perform a data processing operation, and the processing subunit is configured to perform the data processing operation of the unit.
For example, according to a first operating mode of data processing: the data stream sequentially passes through the unit A, D, B, C, the controller configures the execution sequence of the data processing unit, the data stream sequentially passes through the unit A, D, B, C through the bus according to the configured execution sequence, and the unit C outputs the data processing result, that is, the controller first sends the data 1 to be processed to the unit a through the bus, then receives the data 2 processed by the unit a, sends the data 2 processed by the unit a to the unit D, and so on until the data 5 processed by the unit C is received, and the final data processing result can be output through the output unit 305. In actual implementation, the controller may be implemented by a CPU, MPU, or the like located in the apparatus.
It should be noted that all the data processing units a301 in the embodiments of the present invention may further include a selecting subunit a301a and a processing subunit a301B, the data processing unit B302 may further include a selecting subunit B302a and a processing subunit B302B, the data processing unit C303 may further include a selecting subunit C303a and a processing subunit C303B, and the data processing unit D304 may further include a selecting subunit D304a and a processing subunit D304B.
Fig. 4 is a schematic structural diagram of a second operation mode of the data processing unit according to the embodiment of the present invention, in which a solid line may be used to indicate a data signal transmission path, and a dashed line may be used to indicate a data signal transmission disconnection. As shown in fig. 4, the data processing unit a301, the data processing unit B302, the data processing unit C303, the data processing unit D304, and the output unit 305 are connected via a bus to a controller 306 for configuring the execution order of the data processing units. Optionally, each data processing unit is connected according to a preset arrangement order (i.e. from a to B to C to D), and each data processing unit may further include a selection subunit and a processing subunit, where the selection subunit is configured to control the respective data processing unit to perform a data processing operation, and the processing subunit is configured to perform the data processing operation of the unit.
For example, according to a second operating mode of data processing: the data stream sequentially passes through the unit A, D, the controller configures the execution sequence of the data processing unit, and the data stream sequentially passes through the data processing unit A, D through the bus according to the configured execution sequence, that is, the controller first sends the data 1 to be processed to the unit a through the bus, then receives the data 2 processed by the unit a, and sends the data 2 processed by the unit a to the unit D, and receives the data 3 processed by the unit D, and the final data processing result can be output through the output unit 305. Optionally, the controller configures the unit B and the unit C into a transparent transmission state, the controller first sends the data 1 to be processed to the unit a, the unit a directly sends the processed data 2 to the unit D after transparent transmission through the unit B and the unit C, and the unit D outputs the processed data 3 through the output unit 305.
Fig. 5 is a flowchart illustrating a data processing operation executed in the embodiment of the present invention, and as shown in fig. 5, specific steps of executing the data processing operation in the embodiment of the present invention may include:
step 501: acquiring data to be processed;
step 502: configuring a data processing unit according to input data to be processed;
in practical implementation, a mapping list may be established in advance, where the mapping list is used to indicate data processing operations that need to be performed in each data processing unit corresponding to different data processing requirements. And obtaining the data processing operation required to be executed by each data processing unit in the N data processing units by looking up a table according to the input data to be processed.
Further, configuring the corresponding data processing unit according to the obtained data processing operation that each data processing unit needs to execute. It is understood that, when one data processing unit can implement various data processing operations, the data processing operations actually performed by the data processing unit are determined according to data processing requirements.
Step 503: performing data processing operation by using the configured data processing unit;
step 504: and outputting the processed data.
Third embodiment
Based on the foregoing embodiments, in the present embodiment, the data processing method provided by the embodiment of the present invention is further described by taking M data processing main units as an example to perform data processing operations. Wherein M is an integer greater than 1.
In actual implementation, in order to facilitate management and configuration of the data processing units, a plurality of data processing units may be divided into M data processing main units in advance, and data processing is performed by configuring the execution order of the data processing main units and the execution order of the data processing units inside each data processing main unit.
Assuming that the above M data processing main units are 3 data processing main units, different data processing operations are implemented by configuring the execution sequence of the 3 data processing main units in the present embodiment, and the 3 data processing main units include: the data processing system comprises a data processing main unit X, a data processing main unit Y and a data processing main unit Z, wherein each data processing main unit also comprises one or more combinations of a data processing unit A, a data processing unit B, a data processing unit C and a data processing unit D. For example, the data processing unit included in the data processing main unit X may be { a, B, C, D }; the data processing unit contained in the master unit Y may be { a, C }; the data processing unit comprised by the master unit Z may be { a, C, D }.
It should be noted that the functions of the data processing unit a in the data processing main unit are consistent, different data processing operation types can be reasonably divided into different data processing units according to the resource occupation and the programmable principle, the programmable requirement is realized by performing software configuration on the data processing units, and the expandability of data processing can be improved by reserving the same data processing unit. The functions of the data processing unit B are also identical, the functions of the data processing unit C are also identical, and the functions of the data processing unit D are also identical.
Fig. 6 is a schematic flowchart of a third embodiment of a data processing method in an embodiment of the present invention, and as shown in fig. 6, the method includes:
step 601: and acquiring data to be processed.
Step 602: according to the data processing requirements of the data to be processed, determining a data processing main unit for executing the data processing operation, an execution sequence of the data processing main unit when the data processing operation is executed, and an execution sequence of the data processing units in each data processing main unit.
Step 603: configuring the determined data processing main units according to the determined execution sequence of the data processing main units.
In practical implementation, the plurality of data processing main units may include the same data processing unit, and the same data processing unit is used for executing the same data processing operation; accordingly, there is at least one data processing main unit among the plurality of data processing main units that contains the same data processing unit, and/or the same data processing unit is contained in a different data processing main unit.
For example, the data processing master units X, Y and Z may require different execution sequences in different application scenarios, and 3 data processing master units may implement 6 operation modes (i.e. 6 execution sequences), which specifically includes the following:
the data flow in the working mode 1 passes X, Y, Z in sequence;
in the working mode 2, the data flow sequentially passes through X, Z, Y;
the data flow in the working mode 3 passes Y, X, Z in sequence;
in the working mode 4, data flow passes through Y, Z, X in sequence;
in the working mode 5, the data flow passes through Z, X, Y in sequence;
the data flow in operational mode 6 is in turn through Z, Y, X.
Fig. 7 is a schematic structural diagram of a third operating mode of data processing in the embodiment of the present invention, and as shown in fig. 7, the data processing main unit X701, the data processing main unit Y702, and the data processing main unit Z703 are connected to the controller 306 through a bus, and the controller is configured to configure an execution sequence of the data processing units. Optionally, each data processing main unit may further be connected according to a preset arrangement order (i.e., from X to Y to Z), and each data processing main unit may further include a selection unit and a processing unit (i.e., the data processing main unit X701 includes a selection unit X701a and a processing unit X701b, the data processing main unit Y702 includes a selection unit Y702a and a processing unit Y702b, and the data processing main unit Z703 includes a selection unit Z703a and a processing unit Z703b), where the selection unit is configured to control the respective data processing main unit to perform a data processing operation, and the processing unit is configured to perform the data processing operation of the main unit. For example, according to the operating mode of data processing: the data flow passes through the master unit X, Y, Z in turn, the controller configures the execution order of the data processing master units, the data flow passes through the master unit X, Y, Z in turn via the bus according to the configured execution order, and the data processing result is output by the master unit Z.
Step 604: each data processing main unit is configured according to the determined execution sequence of the data processing units in each data processing main unit.
For example, fig. 8 is a schematic diagram of a composition structure of the data processing main unit X in the embodiment of the present invention, and as shown in fig. 8, each data processing unit in the data processing main unit X is connected to the controller 306 through a bus, and the controller is configured to configure an execution sequence of the data processing units. Optionally, each data processing unit may further include a selection subunit and a processing subunit, where the selection subunit is configured to control the respective data processing unit to perform a data processing operation, and the processing subunit is configured to perform the data processing operation of the unit. In the embodiment of the present invention, the data processing main unit X includes a data processing unit a301, a repeating unit a801, a data processing unit B302, a repeating unit B802, a data processing unit C303, a repeating unit C803, a data processing unit D304, and a repeating unit D804. The repeating unit a801 includes 3 repeated data processing units a301, the repeating unit B802 includes 3 repeated data processing units B302, the repeating unit C803 includes 3 repeated data processing units C303, and the repeating unit D801 includes 3 repeated data processing units D304. After configuring the operation mode of the data processing main unit, the execution sequence of the data processing units inside the main unit X is further configured by software.
Fig. 9 is a schematic diagram of a configuration of the data processing main unit Y in the embodiment of the present invention, and as shown in fig. 9, each data processing unit in the data processing main unit Y is connected to the controller 306 through a bus, and the controller is configured to configure an execution sequence of the data processing units. Each data processing unit can also be connected in turn, and each data processing unit can also comprise a selection subunit and a processing subunit, wherein the selection subunit is used for controlling the respective data processing unit to execute data processing operation, and the processing subunit is used for executing the data processing operation of the unit. In the embodiment of the present invention, the data processing main unit Y includes a data processing unit a301, a repeating unit a901, a data processing unit C303, and a repeating unit C902. The repeating unit a901 includes 2 repeating data processing units a301, and the repeating unit C902 includes 2 repeating data processing units C303. After configuring the operation mode of the data processing main unit, the execution sequence of the data processing units inside the main unit Y is further configured by software.
Fig. 10 is a schematic diagram of a configuration of a data processing main unit Z in an embodiment of the present invention, and as shown in fig. 10, each data processing unit in the data processing main unit Z is connected to a controller 306 through a bus, and the controller is configured to configure an execution sequence of the data processing units. Each data processing unit can also be connected in turn, and each data processing unit can also comprise a selection subunit and a processing subunit, wherein the selection subunit is used for controlling the respective data processing unit to execute data processing operation, and the processing subunit is used for executing the data processing operation of the unit. In the embodiment of the present invention, the data processing main unit Z includes a data processing unit a301, a repeating unit a1001, a data processing unit C303, a repeating unit C1002, a data processing unit D304, and a repeating unit D1003. The repeating unit a1001 includes 1 data processing unit a301, the repeating unit C1002 includes 1 data processing unit C303, and the repeating unit D1003 includes 2 repeated data processing units D304. After configuring the operation mode of the data processing main unit, the execution sequence of the data processing units inside the main unit Z is further configured by software.
Step 605: and carrying out data processing on the data to be processed.
In the embodiment of the invention, the execution sequence of the data processing main unit and the execution sequence of the data processing unit in each data processing main unit can be configured according to different data processing requirements to execute different data processing operations, thereby realizing the programmability of data processing; the data processing expandability is realized by reserving partial data processing units in the repeating units in the data processing main unit, and the upgrading operation of the data processing is supported.
Fourth embodiment
Based on the same inventive concept, the embodiment of the invention also provides data processing equipment. Fig. 11 is a schematic diagram of a hardware structure of a data processing apparatus in an embodiment of the present invention, and as shown in fig. 11, the apparatus 11 includes: a processor 1101 and N data processing units 1102; wherein the processor 1101 is configured to perform the following steps: acquiring data to be processed; determining N data processing units 1102 executing data processing operation according to the data processing requirement of the data to be processed, and determining the execution sequence of the N data processing units 1102 when executing the data processing operation; each data processing unit is a functional unit for performing a type of data processing operation, N is an integer greater than 1; and performing data processing on the data to be processed according to the execution sequence of the N data processing units when executing the data processing operation.
It should be noted that the processor is consistent with the controller in one or more of the foregoing method embodiments.
In practical implementation, the processor is specifically configured to perform the following steps: for the ith data processing unit of the N data processing units, when i is 1, 2, 3, …, N-1, executing the following steps: sending the data to be processed of the ith data processing unit to the ith data processing unit for data processing; receiving processed data from an ith data processing unit; determining the (i + 1) th data processing unit according to the execution sequence; and sending the processed data to the (i + 1) th data processing unit for data processing.
In practical implementation, the processor is specifically configured to perform the following steps: for the ith data processing unit of the N data processing units, when i is 1, 2, 3, …, N-1, executing the following steps: sending the data to be processed of the ith data processing unit to the ith data processing unit for data processing; determining the (i + 1) th data processing unit according to the execution sequence; and controlling the (i + 1) th data processing unit to perform data processing on the processed data of the (i) th data processing unit.
In practical implementation, before the step of controlling the i +1 th data processing unit to perform data processing on the processed data of the i-th data processing unit, the processor is further configured to perform the following steps: determining that K data processing units exist between the ith data processing unit and the (i + 1) th data processing unit according to the execution sequence, wherein K is an integer greater than 0; and configuring each data processing unit in the K data processing units into a transparent transmission state, wherein the transparent transmission state is used for indicating each data processing unit in the K data processing units to transmit input data.
In practical applications, the processor may be at least one of an application specific integrated circuit ASIC, DSPD, PLD, FPGA, DSP, CPU, controller, microcontroller, and microprocessor. It will be appreciated that the electronic devices used to implement the processor functions described above may be other devices, and embodiments of the present invention are not limited in particular.
Fifth embodiment
Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium, such as a memory, comprising a computer program, which is executable by a processor of a data processing device to perform the method steps in one or more of the aforementioned embodiments.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.
Claims (3)
1. A data processing method is characterized in that the method is applied to a network processing chip, the network processing chip comprises N data processing units, and N is an integer greater than 1; each data processing unit is a functional unit for performing a type of data processing operation, the data processing operation comprising: insert, delete, modify, operate on, and mark;
the method comprises the following steps:
acquiring data to be processed, wherein the data to be processed is in a message form;
determining N data processing units executing data processing operation and the execution sequence of the N data processing units when executing the data processing operation according to the data processing requirement of the data to be processed;
performing data processing on the data to be processed according to the execution sequence;
the data processing on the data to be processed according to the execution sequence comprises the following steps:
for the ith data processing unit of the N data processing units, when i is 1, 2, 3, …, N-1, executing the following steps:
sending the data to be processed of the ith data processing unit to the ith data processing unit for data processing;
determining the (i + 1) th data processing unit according to the execution sequence;
controlling the (i + 1) th data processing unit to perform data processing on the processed data of the (i) th data processing unit;
before the controlling the i +1 th data processing unit to perform data processing on the processed data of the i-th data processing unit, the method further includes:
determining that K data processing units exist between the ith data processing unit and the (i + 1) th data processing unit according to the execution sequence, wherein K is an integer greater than 0;
and configuring each data processing unit in the K data processing units into a transparent transmission state, wherein the transparent transmission state is used for indicating each data processing unit in the K data processing units to transmit input data.
2. A data processing device, characterized in that the device comprises a network processing chip comprising a processor and N data processing units; wherein the content of the first and second substances,
the processor is configured to perform the steps of: acquiring data to be processed; determining N data processing units for executing data processing operation according to the data processing requirement of the data to be processed, and determining the execution sequence of the N data processing units when executing the data processing operation; each data processing unit is a functional unit for performing a type of data processing operation, the data processing operation including insertion, deletion, modification, operation, and marking, N being an integer greater than 1; performing data processing on the data to be processed according to the execution sequence of the N data processing units when executing data processing operation;
the processor is specifically configured to perform the steps of: for the ith data processing unit of the N data processing units, when i is 1, 2, 3, …, N-1, executing the following steps:
sending the data to be processed of the ith data processing unit to the ith data processing unit for data processing;
determining the (i + 1) th data processing unit according to the execution sequence;
controlling the (i + 1) th data processing unit to perform data processing on the processed data of the (i) th data processing unit;
before executing the step of controlling the i +1 th data processing unit to perform data processing on the processed data of the i-th data processing unit, the processor is further configured to execute the following steps: determining that K data processing units exist between the ith data processing unit and the (i + 1) th data processing unit according to the execution sequence, wherein K is an integer greater than 0;
and configuring each data processing unit in the K data processing units into a transparent transmission state, wherein the transparent transmission state is used for indicating each data processing unit in the K data processing units to transmit input data.
3. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method as claimed in claim 1.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710439303.7A CN109032704B (en) | 2017-06-12 | 2017-06-12 | Data processing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710439303.7A CN109032704B (en) | 2017-06-12 | 2017-06-12 | Data processing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109032704A CN109032704A (en) | 2018-12-18 |
| CN109032704B true CN109032704B (en) | 2022-08-09 |
Family
ID=64628621
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710439303.7A Active CN109032704B (en) | 2017-06-12 | 2017-06-12 | Data processing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109032704B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110377356B (en) * | 2019-06-14 | 2022-09-20 | 北京奇艺世纪科技有限公司 | Task processing method, device and system and computer readable storage medium |
| CN111176725B (en) | 2019-12-27 | 2022-05-06 | 北京市商汤科技开发有限公司 | Data processing method, device, equipment and storage medium |
| CN113360448B (en) * | 2021-06-04 | 2023-04-07 | 展讯通信（上海）有限公司 | Data packet processing method and device |
| CN113596472B (en) * | 2021-07-27 | 2023-12-22 | 安谋科技（中国）有限公司 | Data processing method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010100970A1 (en) * | 2009-03-03 | 2010-09-10 | Canon Kabushiki Kaisha | Data processing apparatus, method for controlling data processing apparatus, and program |
| CN103346974A (en) * | 2013-06-03 | 2013-10-09 | 华为技术有限公司 | Controlling method of service process and network device |
| CN103516684A (en) * | 2012-06-26 | 2014-01-15 | 中兴通讯股份有限公司 | Method for processing data packet and device thereof |
| CN103518360A (en) * | 2012-10-26 | 2014-01-15 | 华为终端有限公司 | Multiple service treatment methods and browser |
| CN105224396A (en) * | 2015-10-14 | 2016-01-06 | 深圳市金证科技股份有限公司 | A kind of business data processing method and device |
| CN106227608A (en) * | 2016-08-30 | 2016-12-14 | 苏州品诺维新医疗科技有限公司 | A kind of data processing method, terminal and system |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| SG132507A1 (en) * | 2001-02-16 | 2007-06-28 | Sony Corp | Data processing method and its apparatus |
| JP4670158B2 (en) * | 2001-02-16 | 2011-04-13 | ソニー株式会社 | Data processing method and semiconductor circuit |
| CN103365804B (en) * | 2012-03-28 | 2016-09-21 | 中国科学院空间科学与应用研究中心 | A kind of read-write control device for chip BU-65170 |
| CN106105153A (en) * | 2014-04-17 | 2016-11-09 | 华为技术有限公司 | A kind of data processing method, control method and device |
| CN106687920B (en) * | 2014-09-02 | 2020-07-03 | 起元科技有限公司 | Management task invocation |
| CN106127302A (en) * | 2016-06-23 | 2016-11-16 | 杭州华为数字技术有限公司 | Process the circuit of data, image processing system, the method and apparatus of process data |
-
2017
- 2017-06-12 CN CN201710439303.7A patent/CN109032704B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2010100970A1 (en) * | 2009-03-03 | 2010-09-10 | Canon Kabushiki Kaisha | Data processing apparatus, method for controlling data processing apparatus, and program |
| CN103516684A (en) * | 2012-06-26 | 2014-01-15 | 中兴通讯股份有限公司 | Method for processing data packet and device thereof |
| CN103518360A (en) * | 2012-10-26 | 2014-01-15 | 华为终端有限公司 | Multiple service treatment methods and browser |
| CN103346974A (en) * | 2013-06-03 | 2013-10-09 | 华为技术有限公司 | Controlling method of service process and network device |
| CN105224396A (en) * | 2015-10-14 | 2016-01-06 | 深圳市金证科技股份有限公司 | A kind of business data processing method and device |
| CN106227608A (en) * | 2016-08-30 | 2016-12-14 | 苏州品诺维新医疗科技有限公司 | A kind of data processing method, terminal and system |
Non-Patent Citations (1)
| Title |
|---|
| Adaptive Distributed Data Stream Management System;Mahmoud Sami Soliman;《中国博士学位论文全文数据库 信息科技辑》;20120115;I138-31 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109032704A (en) | 2018-12-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109032704B (en) | Data processing method and device | |
| CN108293022B (en) | Method, device and system for transmitting message | |
| JP7481436B2 (en) | Method, device and system for forwarding packets in a SR network | |
| CN105389278B (en) | A kind of slave serial communication method based on CAN bus | |
| CN114640616A (en) | Message transmission, segment list generation and compressed segment identification acquisition method and device | |
| CN113535384B (en) | Cloning method and device for application | |
| CN113364638A (en) | Method, electronic device and storage medium for EPA networking | |
| CN112737995B (en) | Method, device and equipment for processing Ethernet frame and storage medium | |
| US10187708B2 (en) | Data processing method and apparatus | |
| EP2367323A1 (en) | Audio signal processor and audio signal processing system | |
| CN109634902B (en) | Data transmission method, device and system suitable for PCIE bus | |
| CN108306757B (en) | Programmable data plane virtual layer construction method and storage medium | |
| Doering et al. | Retrofitting SDN to classical in-vehicle networks: SDN4CAN | |
| CN111245790A (en) | Bit-by-bit configuration method and device of message data, storage medium and electronic equipment | |
| EP3190751A1 (en) | Method and device for delivering forwarding parameters | |
| CN108574637B (en) | Address self-learning method and device and switch | |
| WO2018159677A1 (en) | Data processing device, network system, packet order control circuit and data processing method | |
| CN107454021B (en) | Communication method and device | |
| CN110971716B (en) | Address configuration method, device, system and computer readable storage medium | |
| CN113835467B (en) | Clock synchronization system, method, electronic equipment and storage medium | |
| CN114520839B (en) | Data forwarding method, controller and data forwarding device | |
| CN112804716B (en) | Configuration method and device of data limit value and storage medium | |
| JP5168182B2 (en) | I / O data transfer method | |
| CN110798508A (en) | Server cluster software updating method, device, server and medium | |
| CN105024932A (en) | Traffic policing method and apparatus based on token bucket |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |