CN112612668A - Data processing method, device and computer readable medium - Google Patents
Data processing method, device and computer readable medium Download PDFInfo
- Publication number
- CN112612668A CN112612668A CN202011547990.2A CN202011547990A CN112612668A CN 112612668 A CN112612668 A CN 112612668A CN 202011547990 A CN202011547990 A CN 202011547990A CN 112612668 A CN112612668 A CN 112612668A
- Authority
- CN
- China
- Prior art keywords
- data
- packet
- sub
- transmitted
- sections
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 26
- 230000005540 biological transmission Effects 0.000 claims abstract description 12
- 238000000034 method Methods 0.000 claims description 29
- 238000012545 processing Methods 0.000 claims description 17
- 230000008859 change Effects 0.000 claims description 10
- 230000011218 segmentation Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 4
- 230000008569 process Effects 0.000 description 12
- 238000005192 partition Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000000644 propagated effect Effects 0.000 description 3
- 238000000638 solvent extraction Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000012634 fragment Substances 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/22—Detection or location of defective computer hardware by testing during standby operation or during idle time, e.g. start-up testing
- G06F11/26—Functional testing
- G06F11/263—Generation of test inputs, e.g. test vectors, patterns or sequences ; with adaptation of the tested hardware for testability with external testers
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention provides a data processing method, which comprises the following steps: dividing data to be transmitted into n sections and configuring the number of half bytes of each section, wherein n is a positive integer; combining the n small sections according to x sections respectively, wherein each section comprises one or more sections, and x is a positive integer; based on each section type, generating a plurality of different data packets corresponding to the data to be transmitted according to a set rule, and calculating the total half byte number of each data packet; the packet with the minimum total half byte count is selected for transmission.
Description
Technical Field
The present invention relates to the field of information technologies, and in particular, to a data processing method, apparatus, and computer readable medium.
Background
In the development of various soc (system On chip) systems, debugging of the system is a very critical step. CoreSight is an architecture proposed by ARM corporation for implementing debug (debug) and data trace (trace) on complex socs. The architecture includes a plurality of CoreSight components. A plurality of CoreSight components form a CoreSight system.
In the development process of a System on chip (SoC), multi-core tracking is required to be realized, so that the data volume is huge, the data to be tracked can be compressed to a certain extent, the transmission bandwidth can be effectively reduced, the compressed data is packaged according to STP (System Trace Protocol) v2 version Protocol, and the packaged data is transmitted to a CoreSight System through an ATB bus Protocol port. The data is transmitted through a coresight component, and finally transmitted to an off-chip analyzer through a TPIU (trace port interface unit) interface for analysis and subsequent debugging.
In trace data compression, only a fixed data bit width is selected according to STP (version, e.g., series V2). According to the specification of the STP protocol V2 version series, the optional ranges of the packets are: d4(M), D8(M), D16(M), D32(M), and D64 (M). D4(M), D8(M), D16(M), D32(M), D64(M) respectively represent packets including data of 4 bits, 8 bits, 16 bits, 32 bits, and 64 bits and an operation code as a flag, for example: when the transmitted 32-bit data is 0x001FFFFF, 21 bits are actually required to be transmitted after the high order 0 is removed, but due to the limitation of the STP protocol V2 series version protocols, the D32 data packet still needs to be used in the transmission, so that 11 bits are wasted, and the maximum compression strength is not reached.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a data processing method, which can improve the data processing efficiency.
In order to solve the above technical problem, the present invention provides a data processing method, comprising the following steps:
dividing data to be transmitted into n sections and configuring the number of half bytes of each section, wherein n is a positive integer; combining the n small sections according to x sections respectively, wherein each section comprises one or more sections, and x is a positive integer; based on each section type, generating a plurality of different data packets corresponding to the data to be transmitted according to a set rule, and calculating the total half byte number of each data packet; the packet with the minimum total half byte count is selected for transmission.
In an embodiment of the present invention, the data to be transmitted is obtained based on combined data, and the combined data is formed by splicing a plurality of types of data according to a specific splicing manner.
In an embodiment of the invention, the combined data has a high end and a low end, and the specific splicing manner includes placing the data with the lowest frequency of change at the high end of the combined data.
In an embodiment of the present invention, the step of obtaining the data to be transmitted based on the combined data includes: performing logic operation on the current combined data and the previous combined data to obtain the data to be transmitted
In an embodiment of the present invention, the step of dividing the data to be transmitted into n sections and configuring the number of half bytes of each section includes: and arranging the continuously changed bits in the same section, and placing the continuously unchanged bits in the same section.
In an embodiment of the present invention, the step of generating different data packets corresponding to the data to be transmitted according to a set rule based on the divided segments and sections, and calculating a total number of half bytes of each data packet includes: when the segmented patterns formed by division comprise a plurality of segments, inserting null data packets between each segment pattern, wherein the number of nibbles of each null data packet is 1, and the data of each bit in each null data packet is zero; converting each segmented data into a sub data packet in a data packet, wherein the sub data packet comprises a packet header and a packet body, the packet body comprises the converted data, the packet header comprises an operation code, and the operation code corresponds to the converted data; and forming the data packet based on the sub data packet, and calculating the total half byte number of the data packet.
In an embodiment of the invention, the specific splicing manner further includes placing data with the highest frequency of change at a lower end of the combined data.
In an embodiment of the present invention, the lower the frequency of change of data, the higher the rank order in the combined data.
In an embodiment of the present invention, the step of converting the data of each segment into a sub-packet in the data packet comprises: when the data of each bit in the segment corresponding to the non-highest bit is all zero, the segment does not form a data packet; when the data of each bit in the section corresponding to the highest bit is all zero, forming a corresponding sub-data packet in a D (4) M mode; when the data of all the bits in the segment corresponding to the highest bit is not all zero and the segment corresponding to the highest bit comprises t bits, forming a corresponding sub-data packet in a mode of D (t) M; d (4) M and D (t) M represent that the packet represents the converted data by 4 and t bits, the packet header comprises an operation code for marking the end of the conversion of the data, and t is a positive integer; removing zero data in data bits higher than the highest non-zero data bit in each segment except the segment corresponding to the highest bit to form data to be converted; determining to form corresponding sub-packets in a mode of D4, D8, D16, D32 and D64 based on data bits contained in the data to be converted; d4, D8, D16, D32, and D64 respectively indicate that the packet represents a sub-packet in which the converted data is characterized by 4, 8, 16, 32, and 64 bits, the packet header of the corresponding sub-packet formed in the manner of D4, D8, D16, D32, and D64 includes an operation code corresponding to the data in the packet body, and the number of half bytes of the operation code is 1.
In an embodiment of the present invention, the data processing method further includes analyzing the data packet with the minimum total half byte number after receiving the transmitted data packet with the minimum total half byte number; the step of resolving comprises: and analyzing the data packet according to the division setting for dividing the segment and the division setting for the section.
In an embodiment of the present invention, the data processing method further includes analyzing the data packet with the minimum total half byte number after receiving the transmitted data packet with the minimum total half byte number; the step of resolving comprises: restoring the number of the sections of the data according to the number of the empty data packets in the data packets; restoring a sub data packet according to the division setting of the segments, the division setting of the sections, the operation code in the packet header in the sub data packet of each segment and the converted data in the packet body; restoring a data packet based on the restored sub data packet; when two continuous null data packets appear in the received data packet with the minimum total half byte number, restoring a sub data packet comprising zero data according to the segmentation setting of the segments and the segmentation setting of the sections.
The present invention also provides a data processing apparatus comprising: a memory for storing instructions executable by the processor; and a processor for executing the instructions to implement the method of any of the preceding claims.
The invention also provides a computer readable medium having stored thereon computer program code which, when executed by a processor, implements the method of any of the preceding claims.
Compared with the prior art, the invention has the following advantages: the data packet with the minimum total half byte number is obtained by splicing various data and dividing the spliced data into small sections and sections, and is transmitted, so that the data transmission and processing are realized by using bit resources as few as possible.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principle of the invention. In the drawings:
fig. 1 is a flowchart of a data processing method according to an embodiment of the present application.
Fig. 2 is a flowchart of a step of generating different data packets corresponding to the data to be transmitted according to a set rule and calculating a total number of half bytes of each data packet in the data processing method according to an embodiment of the present application.
FIG. 3 is a flowchart illustrating steps for converting each segmented data into subpackets in a data packet, according to one embodiment of the present application.
Fig. 4 is a process diagram of data processing according to an embodiment of the present application.
Fig. 5 is a flowchart of a parsing process of data processing according to an embodiment of the present application.
Fig. 6 is a schematic diagram of a process of parsing data in the data processing method according to an embodiment of the present application.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described herein, and thus the present invention is not limited to the specific embodiments disclosed below.
As used herein, the terms "a," "an," "the," and/or "the" are not intended to be inclusive and include the plural unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that steps and elements are included which are explicitly identified, that the steps and elements do not form an exclusive list, and that a method or apparatus may include other steps or elements.
The relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present application unless specifically stated otherwise. Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description. Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate. In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
Flow charts are used herein to illustrate operations performed by systems according to embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in the exact order in which they are performed. Rather, various steps may be processed in reverse order or simultaneously. Meanwhile, other operations are added to or removed from these processes.
Embodiments of the present application describe a data processing method, apparatus, and computer readable medium.
Fig. 1 is a flowchart of a data processing method according to an embodiment of the present application. As shown in fig. 1, the data processing method includes, step 101, dividing the data to be transmitted into n sections and configuring the number of half bytes of each section. And 102, combining the n sections according to x sections, wherein each section comprises one or more sections. 103, based on each section, generating a plurality of different data packets corresponding to the data to be transmitted according to a set rule, and calculating the total half byte number of each data packet. At step 104, the packet with the minimum total half byte number is selected for transmission.
In step 101, data to be transmitted is divided into n sections (seg) and the number of half bytes (number of nibbles) of each section is configured, where n is a positive integer. In an embodiment, the step of dividing the data to be transmitted into n sections and configuring the number of half bytes of each section includes: and arranging the continuously changed bits in the same section, and placing the continuously unchanged bits in the same section.
In some embodiments, the data to be transmitted may be derived based on the combined data. The data to be transmitted may also be data in which the different fields themselves are formed separately. In the case of combined data, multiple data may be spliced to form combined data according to a particular splicing manner. The various data are, for example, data of different fields that need to be tracked during the operation of the SoC system, such as data of different fields of types of ID number, memory address, data length, trigger signal, control signal, and the like. These data are traced (trace) for subsequent monitoring and function debugging of the system. In an embodiment, different types of command data in the AXI bus protocol data are tracked, for example. Specifically, commands to be tracked include awtens, awsize, awaddr, awburst, awwid, and the like. As mentioned above, if the STP protocol is directly used to trace data, only a fixed data bit width can be selected. For example, if the awaddr data is 40 bits, if the STP protocol is directly used, a data packet in D64(M) format is required for transmission. At this time, more transmission resources are wasted.
After the multiple data are spliced to form combined data according to a specific splicing mode, the data to be transmitted are obtained based on the combined data. In an embodiment, the step of obtaining the data to be transmitted based on the combined data includes, for example, performing a logical operation on the current combined data (i.e., the combined data formed after splicing) and the combined data transmitted last time to obtain the data to be transmitted. The logical operation is, for example, an exclusive or operation.
In connection with the logic operation, when the data are spliced to form the combined data, the data may also be spliced according to a specific splicing rule. For example, assuming that the combined data has a high end and a low end, the specific splicing manner may include placing the data with the lowest frequency of change at the high end of the combined data. In other embodiments, the way of splicing to form the combined data further includes, for example, placing the data with the highest frequency of change at the lower end of the combined data. Further, the splicing rule may be set such that the lower the frequency of change of data, the higher the rank order in the combined data. For example, if the type of the logical operation is xor, when data is spliced, data with the lowest change frequency is placed at the high-order end of the combined data, and after xor, some data to be transmitted with zero bits will be obtained based on unchanged data bits compared with data transmitted at the previous time.
In step 102, the n subsections are combined according to x sections respectively, each section comprises one or more sections, wherein x is a positive integer. The value of x can be set according to actual requirements.
In step 103, based on each segment formed by partitioning, a plurality of different data packets corresponding to the data to be transmitted are generated according to a set rule, and the total half byte number of each data packet is calculated.
In some embodiments, the step of generating different data packets corresponding to the data to be transmitted according to a set rule based on the segments and sections that are divided, and calculating the total number of half bytes of each data packet includes step 201, when the segment formula formed by the division includes a plurality of segments, inserting null data packets (null packets) between each segment formula, where the number of half bytes of each null data packet is 1, and data of each bit in each null data packet is zero. Step 202, converting each piece of segmented data into a sub-data packet in a data packet, where the sub-data packet includes a packet header and a packet body, the packet body includes the converted data, the packet header includes an operation code, and the operation code corresponds to the converted data. Step 203, forming the data packet based on the sub data packet.
In step 202, the step of converting the data of each segment into sub-data packets in the data packet includes, step 301, when the data of each bit in the segment corresponding to the non-highest bit is all zero, the segment does not form a data packet; when the data of each bit in the section corresponding to the highest bit is all zero, forming a corresponding sub-data packet in a D (4) M mode; and when the data of all the bits in the segment corresponding to the highest bit is not all zero and the segment corresponding to the highest bit comprises t bits, forming a corresponding sub-data packet in a mode of D (t) M. D (4) M and D (t) M represent that the packet represents the converted data by 4 and t bits, the packet header comprises an operation code for marking the end of the conversion of the data, and t is a positive integer. The packet header comprises the segment corresponding to the operation code which marks the end of the conversion of the data and is converted to the highest bit.
To show the data processing method of the present application more specifically, for example, in step 101, the current combined data (curr _ data) formed by splicing according to a specific splicing manner is:
curr_data＝0x12345677_12345678_12345677_12345678
let the last combined data (last _ data) be:
last_data＝0x12345678_12345678_12345678_12345678
and performing exclusive or on the current combined data and the previous combined data to obtain a result M:
M＝curr_data⊕last_data＝0x000000F_00000000_0000000F_00000000
and M is the obtained data to be transmitted.
Next, in step 101, the data to be transmitted is divided into n sections and the number of half bytes per section is configured. In step 102, the n subsections are combined according to x sections respectively, wherein each section comprises one or more sections.
In this example, in step 101, it is assumed that data M that needs to be compressed is configured as 8 sections (seg). For simplicity, each measure is assumed to be 4 nibbles (nibbles).
Then, in step 102, the n segments are combined according to x segment types, each segment type including one or more segments. For example, in the present example, it is assumed that the combination is performed in 8 stages.
Fig. 4 is a process diagram of data processing according to an embodiment of the present application. The above-described combination in 8 stages is, for example, a free combination method as indicated by a region 401 in fig. 4. The area 401 of fig. 4 shows 8 combinations corresponding to the segment types respectively. In the actual dividing, the number of the segment formed by dividing can be determined according to the requirement. For example, the method may be a 6-stage method, a 10-stage method, a 12-stage method, or the like. After the segment type formed by division is determined, for some division modes which are used more frequently, the division modes can also be summarized into division programs. For example, the 8-segment equation in fig. 4 can be summarized as:
segment (part1) of segment 1 is composed of all subsections (seg);
the first segment (part1) in the 2-segment formula consists of the first half of the measure (seg), and the second segment (part2) consists of the second half of the measure (seg);
and 3-8 section type combination modes are as follows:
if the combination needs to be combined into an x segment (x is more than or equal to 3 and less than or equal to 8), when x is an odd number, y is equal to x, and when x is an even number, y is equal to x-1, and z is (8/y) rounded downwards;
part 2＝seg(z+1)+…+seg[2z]
……
part(x-1)＝seg[(x-2)z+1]+…+seg[(x-1)z]
part x＝seg[(x-1)z+1]+…+seg n
next, in step 103, based on each segment, a plurality of different data packets corresponding to the data to be transmitted are generated according to a set rule, and the total half byte number of each data packet is calculated. Specifically, according to the foregoing steps 201 to 203, a plurality of different packets are obtained, as indicated by an area 402 in fig. 4. Then, the total half byte number (i.e., the total nibble number) of the different data packets is calculated, and the total half byte number (the total nibble number) of the data packets corresponding to the various segments of the present embodiment is as indicated by a region 403 in fig. 4.
Then, in step 104, the packet with the minimum total half byte count is selected for transmission. In fig. 4, for example, after passing through a comparator 404, the minimum total number of nibbles (total nibble number) is obtained, and the data packet corresponding to the minimum total number of nibbles is determined, in fig. 4, for example, the data packet is formed in a 4-segment division manner as indicated by 405, which is the processed data, and the processed data can be transmitted. Since the data at this time has the minimum total number of half bytes (total number of nibbles), the transmission of the data can be realized with the minimum bit resources.
In an embodiment of the application, the method further includes parsing the minimum total half byte number of the data packet after receiving the transmitted data packet with the minimum total half byte number. Specifically, the packet is parsed based on a partition setting in which the segments are formed by partitioning and a partition setting in which the sections are formed by partitioning. Fig. 5 is a flowchart of a parsing process of data processing according to an embodiment of the present application.
More specifically, for the data packet with the minimum total half byte number after the steps 101 to 104, and the detailed processing steps 201 to 203, and 301 to 303, after receiving the transmitted data packet with the minimum total half byte number, the step of parsing the data packet with the minimum total half byte number to obtain the data before processing includes a step 501 of restoring the number of segments of the data according to the number of null data packets in the data packet. Step 502, restoring a sub-packet according to the partition setting of the segment, the partition setting of the section, the operation code in the packet header of each sub-packet of the segment, and the converted data included in the packet body. When two continuous null data packets appear in the received data packet with the minimum total half byte number, restoring a sub data packet comprising zero data according to the segmentation setting of the segments and the segmentation setting of the sections. And step 503, restoring a data packet based on the restored sub data packet.
As shown in fig. 6, the process of parsing and restoring the processed data "FD 00FC 0" to be transmitted corresponding to the four-segment data packet is obtained in the specific example described in fig. 4. Fig. 6 is a schematic diagram of a process of parsing data in the data processing method according to an embodiment of the present application.
Specifically, for the received packet data "FD 00FC 0" with the minimum total half byte number, in step 501, the number of null packets (null packets) therein is obtained, and the number of segments of the data is restored. In "FD 00FC 0", 3 null packets are included, each of which occupies 1 nibble (nibble). Since in the previous data processing null packets were inserted between the two segments. The packet "FD 00FC 0" can be recovered based on 3 null packets and comprises 4 fragments. In fig. 6, regions 601 to 602 are illustrated as blocks, for example. The 'part 1_ enc' in region 602 refers to the first segment that was previously encoded (encode). 'part 2_ enc', 'part 3_ enc', 'part 4_ enc' refer to the second, third and fourth segments, respectively, that are encoded (encode).
In step 502, the sub-packet of each segment (part) is restored based on the partition setting of the segment, the partition setting of the section, the operation code in the packet header in the sub-packet of each segment, and the converted data included in the packet body. The operation code is determined according to a transport protocol, for example the STP protocol. In fig. 6, for example, the areas 602 to 603 are illustrated, or the areas 601 and 602 to 603 are illustrated. The parsing process here is a context-dependent process. Regions 603 illustrate the decoded segments.
Then, in step 503, a data packet is restored based on the restored sub-packets. In fig. 6, the original data M is obtained by, for example, splicing, as indicated by 604 in fig. 6. In an embodiment, the restoring process further includes, for example, performing a logical operation, such as an exclusive or operation, on the data of the data packet obtained by the splicing and the original data of the data packet transmitted last time to obtain the original data corresponding to the data of the data packet received this time, that is, the data formed in the initial splicing step.
According to the technical scheme, various data are spliced, the spliced data are divided into small sections and sectional types, the total half byte number corresponding to different splitting modes is calculated, the data packet with the minimum total half byte number is obtained and transmitted, and therefore transmission and processing of the data are achieved with bit resources as few as possible.
The present invention also provides a data processing apparatus comprising: a memory for storing instructions executable by the processor; and a processor for executing the instructions to implement the method of any of the preceding claims to implement data processing functions.
The invention also provides a computer readable medium having stored thereon computer program code which, when executed by a processor, implements a method as in any one of the preceding claims to implement data processing functions.
Having thus described the basic concept, it will be apparent to those skilled in the art that the foregoing disclosure is by way of example only, and is not intended to limit the present application. Various modifications, improvements and adaptations to the present application may occur to those skilled in the art, although not explicitly described herein. Such modifications, improvements and adaptations are proposed in the present application and thus fall within the spirit and scope of the exemplary embodiments of the present application.
Also, this application uses specific language to describe embodiments of the application. Reference throughout this specification to "one embodiment," "an embodiment," and/or "some embodiments" means that a particular feature, structure, or characteristic described in connection with at least one embodiment of the present application is included in at least one embodiment of the present application. Therefore, it is emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, some features, structures, or characteristics of one or more embodiments of the present application may be combined as appropriate.
Aspects of the present application may be embodied entirely in hardware, entirely in software (including firmware, resident software, micro-code, etc.) or in a combination of hardware and software. The above hardware or software may be referred to as "data block," module, "" engine, "" unit, "" component, "or" system. The processor may be one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), digital signal processing devices (DAPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or a combination thereof. Furthermore, aspects of the present application may be represented as a computer product, including computer readable program code, embodied in one or more computer readable media. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips … …), optical disks (e.g., Compact Disk (CD), Digital Versatile Disk (DVD) … …), smart cards, and flash memory devices (e.g., card, stick, key drive … …).
The computer readable medium may comprise a propagated data signal with the computer program code embodied therein, for example, on a baseband or as part of a carrier wave. The propagated signal may take any of a variety of forms, including electromagnetic, optical, and the like, or any suitable combination. The computer readable medium can be any computer readable medium that can communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or device. Program code on a computer readable medium may be propagated over any suitable medium, including radio, electrical cable, fiber optic cable, radio frequency signals, or the like, or any combination of the preceding.
Similarly, it should be noted that in the preceding description of embodiments of the application, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the embodiments. This method of disclosure, however, is not intended to require more features than are expressly recited in the claims. Indeed, the embodiments may be characterized as having less than all of the features of a single embodiment disclosed above.
Although the present application has been described with reference to the present specific embodiments, it will be recognized by those skilled in the art that the foregoing embodiments are merely illustrative of the present application and that various changes and substitutions of equivalents may be made without departing from the spirit of the application, and therefore, it is intended that all changes and modifications to the above-described embodiments that come within the spirit of the application fall within the scope of the claims of the application.
Claims (13)
1. A method of data processing comprising the steps of:
dividing data to be transmitted into n sections and configuring the number of half bytes of each section, wherein n is a positive integer;
combining the n small sections according to x sections respectively, wherein each section comprises one or more sections, and x is a positive integer;
based on each section type, generating a plurality of different data packets corresponding to the data to be transmitted according to a set rule, and calculating the total half byte number of each data packet;
the packet with the minimum total half byte count is selected for transmission.
2. The data processing method according to claim 1, wherein the data to be transmitted is obtained based on combined data, and the combined data is formed by splicing a plurality of types of data according to a specific splicing manner.
3. The data processing method of claim 2, wherein the combined data has a high end and a low end, and the specific splicing manner includes placing data with the lowest frequency of change at the high end of the combined data.
4. The data processing method of claim 2, wherein the step of deriving the data to be transmitted based on the combined data comprises: and performing logic operation on the current combined data and the previous combined data to obtain the data to be transmitted.
5. The data processing method of claim 1, wherein the step of dividing the data to be transmitted into n sections and configuring the number of half bytes per section comprises: and arranging the continuously changed bits in the same section, and placing the continuously unchanged bits in the same section.
6. The data processing method according to claim 1, wherein the step of generating different data packets corresponding to the data to be transmitted according to a set rule based on the segments and sections divided and calculating the total number of half bytes of each data packet comprises:
when the segmented patterns formed by division comprise a plurality of segments, inserting null data packets between each segment pattern, wherein the number of nibbles of each null data packet is 1, and the data of each bit in each null data packet is zero;
converting each segmented data into a sub data packet in a data packet, wherein the sub data packet comprises a packet header and a packet body, the packet body comprises the converted data, the packet header comprises an operation code, and the operation code corresponds to the converted data;
and forming the data packet based on the sub data packet, and calculating the total half byte number of the data packet.
7. The data processing method of claim 3, wherein the specific splicing manner further comprises placing data with the highest frequency of change at a lower end of the combined data.
8. The data processing method according to claim 7, wherein the lower the frequency of change of data, the higher the rank order in the combined data.
9. The data processing method of claim 6, wherein the step of converting each segmented data into sub-packets in a data packet comprises:
when the data of each bit in the segment corresponding to the non-highest bit is all zero, the segment does not form a data packet; when the data of each bit in the section corresponding to the highest bit is all zero, forming a corresponding sub-data packet in a D (4) M mode; when the data of all the bits in the segment corresponding to the highest bit is not all zero and the segment corresponding to the highest bit comprises t bits, forming a corresponding sub-data packet in a mode of D (t) M; d (4) M and D (t) M represent that the packet represents the converted data by 4 and t bits, the packet header comprises an operation code for marking the end of the conversion of the data, and t is a positive integer;
removing zero data in data bits higher than the highest non-zero data bit in each segment except the segment corresponding to the highest bit to form data to be converted;
determining to form corresponding sub-packets in a mode of D4, D8, D16, D32 and D64 based on data bits contained in the data to be converted;
d4, D8, D16, D32, and D64 respectively indicate that the packet represents a sub-packet in which the converted data is characterized by 4, 8, 16, 32, and 64 bits, the packet header of the corresponding sub-packet formed in the manner of D4, D8, D16, D32, and D64 includes an operation code corresponding to the data in the packet body, and the number of half bytes of the operation code is 1.
10. The data processing method of claim 1, further comprising parsing the minimum total half byte number of data packets after receiving the transmitted data packets having the minimum total half byte number; the step of resolving comprises:
and analyzing the data packet according to the division setting for dividing the segment and the division setting for the section.
11. The data processing method of claim 9, further comprising parsing the minimum total half byte number of data packets after receiving the transmitted data packets having the minimum total half byte number; the step of resolving comprises:
restoring the number of the sections of the data according to the number of the empty data packets in the data packets;
restoring a sub data packet according to the division setting of the segments, the division setting of the sections, the operation code in the packet header in the sub data packet of each segment and the converted data in the packet body;
restoring a data packet based on the restored sub data packet;
when two continuous null data packets appear in the received data packet with the minimum total half byte number, restoring a sub data packet comprising zero data according to the segmentation setting of the segments and the segmentation setting of the sections.
12. A data processing apparatus comprising:
a memory for storing instructions executable by the processor; and
a processor for executing the instructions to implement the method of any one of claims 1-11.
13. A computer-readable medium having stored thereon computer program code which, when executed by a processor, implements the method of any of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011547990.2A CN112612668A (en) | 2020-12-24 | 2020-12-24 | Data processing method, device and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011547990.2A CN112612668A (en) | 2020-12-24 | 2020-12-24 | Data processing method, device and computer readable medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112612668A true CN112612668A (en) | 2021-04-06 |
Family
ID=75244658
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011547990.2A Pending CN112612668A (en) | 2020-12-24 | 2020-12-24 | Data processing method, device and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112612668A (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4211997A (en) * | 1978-11-03 | 1980-07-08 | Ampex Corporation | Method and apparatus employing an improved format for recording and reproducing digital audio |
| US4377862A (en) * | 1978-12-06 | 1983-03-22 | The Boeing Company | Method of error control in asynchronous communications |
| US20040024592A1 (en) * | 2002-08-01 | 2004-02-05 | Yamaha Corporation | Audio data processing apparatus and audio data distributing apparatus |
| CN1745533A (en) * | 2003-02-26 | 2006-03-08 | 艾利森电话股份有限公司 | Transmission control method in an ARQ system |
| US20060259168A1 (en) * | 2003-07-21 | 2006-11-16 | Stefan Geyersberger | Audio file format conversion |
| CN101286819A (en) * | 2008-05-07 | 2008-10-15 | 中兴通讯股份有限公司 | Data receiving method and device |
| US20110029836A1 (en) * | 2009-07-30 | 2011-02-03 | Cleversafe, Inc. | Method and apparatus for storage integrity processing based on error types in a dispersed storage network |
| US20120233523A1 (en) * | 2011-03-10 | 2012-09-13 | Icform, Inc | Programmable Data Storage Management |
| WO2014035497A1 (en) * | 2012-08-28 | 2014-03-06 | Aoptix Technologies, Inc. | Assessment and correction of transmitted data |
| CN106096022A (en) * | 2016-06-22 | 2016-11-09 | 杭州迪普科技有限公司 | The partitioning method and device of multiple domain net bag classifying rules |
| CN106462605A (en) * | 2014-05-13 | 2017-02-22 | 云聚公司 | Distributed secure data storage and transmission of streaming media content |
| US20180107430A1 (en) * | 2014-10-29 | 2018-04-19 | International Business Machines Corporation | Coordinating arrival times of data slices in a dispersed storage network |
-
2020
- 2020-12-24 CN CN202011547990.2A patent/CN112612668A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US4211997A (en) * | 1978-11-03 | 1980-07-08 | Ampex Corporation | Method and apparatus employing an improved format for recording and reproducing digital audio |
| US4377862A (en) * | 1978-12-06 | 1983-03-22 | The Boeing Company | Method of error control in asynchronous communications |
| US20040024592A1 (en) * | 2002-08-01 | 2004-02-05 | Yamaha Corporation | Audio data processing apparatus and audio data distributing apparatus |
| CN1745533A (en) * | 2003-02-26 | 2006-03-08 | 艾利森电话股份有限公司 | Transmission control method in an ARQ system |
| US20060259168A1 (en) * | 2003-07-21 | 2006-11-16 | Stefan Geyersberger | Audio file format conversion |
| CN101286819A (en) * | 2008-05-07 | 2008-10-15 | 中兴通讯股份有限公司 | Data receiving method and device |
| US20110029836A1 (en) * | 2009-07-30 | 2011-02-03 | Cleversafe, Inc. | Method and apparatus for storage integrity processing based on error types in a dispersed storage network |
| US20120233523A1 (en) * | 2011-03-10 | 2012-09-13 | Icform, Inc | Programmable Data Storage Management |
| WO2014035497A1 (en) * | 2012-08-28 | 2014-03-06 | Aoptix Technologies, Inc. | Assessment and correction of transmitted data |
| CN106462605A (en) * | 2014-05-13 | 2017-02-22 | 云聚公司 | Distributed secure data storage and transmission of streaming media content |
| US20180107430A1 (en) * | 2014-10-29 | 2018-04-19 | International Business Machines Corporation | Coordinating arrival times of data slices in a dispersed storage network |
| CN106096022A (en) * | 2016-06-22 | 2016-11-09 | 杭州迪普科技有限公司 | The partitioning method and device of multiple domain net bag classifying rules |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP2668577B1 (en) | Unrolling quantifications to control in-degree and/or out degree of automaton | |
| US9647684B2 (en) | Memory-based history search | |
| RU2629440C2 (en) | Device and method for acceleration of compression and decompression operations | |
| KR100875836B1 (en) | Instruction instruction compression apparatus and method for parallel processing BLU computer | |
| CN105264779A (en) | Data compression and decompression using simd instructions | |
| CN107547574B (en) | Communication system and method based on universal protocol | |
| CN103853650A (en) | Test case generating method and device for fuzz testing | |
| CN1959686A (en) | Methods and system to model arrangements of asynchronous interface | |
| CN110708133B (en) | Method and device for clock synchronization and time synchronization in system based on FPGA | |
| CN111367786B (en) | Symbol execution method, electronic equipment and storage medium | |
| CN110769002A (en) | LabVIEW-based message analysis method, system, electronic device and medium | |
| CN105528183A (en) | Data storage method and storage equipment | |
| KR20210002817A (en) | Method for constructing parity-check concatenated polar codes and apparatus therefor | |
| CN112612668A (en) | Data processing method, device and computer readable medium | |
| CN113238737A (en) | Page packaging method and device, electronic equipment and storage medium | |
| US9197243B2 (en) | Compression ratio for a compression engine | |
| JP2014232523A (en) | Searching apparatus utilizing sub-word finite state machines | |
| CN115544490A (en) | Method and system for detecting password constant in binary file | |
| US10915547B2 (en) | Optimizing data conversion using pattern frequency | |
| CN115599394A (en) | Serialization and deserialization method, device, computer equipment and storage medium | |
| US8589661B2 (en) | Odd and even start bit vectors | |
| CN114490861A (en) | Telemetry data analysis method, device, equipment and medium | |
| CN113890540A (en) | Parallel acceleration LZ77 decoding method and device | |
| CN116016296B (en) | Binary data processing method, system and equipment based on JSON | |
| CN114372582B (en) | Quantum automatic coding method based on machine learning framework and related device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |