CN111602111A - Data processing method and device - Google Patents

Data processing method and device Download PDF

Info

Publication number
CN111602111A
CN111602111A CN201880086748.4A CN201880086748A CN111602111A CN 111602111 A CN111602111 A CN 111602111A CN 201880086748 A CN201880086748 A CN 201880086748A CN 111602111 A CN111602111 A CN 111602111A
Authority
CN
China
Prior art keywords
data
compression
processed
order
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201880086748.4A
Other languages
Chinese (zh)
Inventor
谢俊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Royole Technologies Co Ltd
Original Assignee
Shenzhen Royole Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Royole Technologies Co Ltd filed Critical Shenzhen Royole Technologies Co Ltd
Publication of CN111602111A publication Critical patent/CN111602111A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)

Abstract

The embodiment of the invention discloses a data processing method and a data processing device, wherein the method comprises the following steps: acquiring compression parameters aiming at a data set to be processed, wherein the compression parameters comprise an offset, a maximum compression length and a control frame mark; shifting the data to be processed included in the data set to be processed according to the offset, and dividing the shifted data to be processed into a plurality of data segments to be processed according to the maximum compression length; and respectively compressing the plurality of data segments to be processed, and adding the compressed data into the compressed data arrays according to the time sequence by using the control frame marks corresponding to the compression mode adopted in the compression process. The invention can realize the data compression to reach higher compression ratio with smaller calculation amount.

Description

Data processing method and device Technical Field
The present invention relates to the field of computer technologies, and in particular, to a data processing method and apparatus.
Background
Data compression refers to a method for reducing the amount of data to reduce the storage space without losing information, so as to improve the efficiency of data transmission, storage and processing. In the case of limited network bandwidth and limited bearer capacity, data compression is often required to enable faster transmission or more convenient storage of data. For example, the current multimedia information includes various media information such as text, sound, image, animation, and video, and the data size is very large after the digital processing. Without data compression processing of multimedia data, the computer system cannot store and exchange it. The existing data compression method is generally used for compressing data with large data volume such as images, audios, videos and the like, and the calculation amount is large for achieving a certain data compression rate.
Disclosure of Invention
The embodiment of the invention provides a data processing method and device, which can achieve high compression rate by using small calculation amount for data compression.
A first aspect of an embodiment of the present invention provides a data processing method, including:
and acquiring compression parameters aiming at the data set to be processed, wherein the compression parameters comprise an offset, a maximum compression length and a control frame mark.
The data to be processed included in the data set to be processed is shifted according to the shift amount, and the shifted data to be processed is divided into a plurality of data segments to be processed according to the maximum compression length;
and respectively compressing the plurality of data segments to be processed, and respectively adding the compressed data into the compressed data arrays according to the time sequence by using the control frame marks corresponding to the compression mode adopted in the compression process.
As a possible implementation manner, the compression parameters further include a frame length, a first order frame length, and a second order frame length, and a specific implementation manner for obtaining the compression parameters for the data set to be processed may be: acquiring a value range of to-be-processed data included in a to-be-processed data set; according to the value range of the data to be processed included in the data set to be processed, determining the offset, the frame length, the first-order frame length, the second-order frame length and a control frame mark aiming at the data set to be processed, wherein the control frame mark comprises a starting frame mark, a compression mode mark and an ending frame mark. Wherein the start frame marker is used for marking the start of a compressed data segment in the compressed data array, and the start frame marker is located at the first position of the compressed data segment; the end frame marker is used for marking the end of one compressed data segment in the compressed data array, and the end frame marker is positioned at the last position of the compressed data segment.
As a possible implementation manner, a specific implementation manner for respectively compressing the multiple data segments to be processed may be: determining the current position of an indicating cursor in a target data segment to be processed aiming at the target data segment to be processed in a plurality of data segments to be processed, wherein the indicating cursor is used for indicating the next data segment to be processed, and the target data segment to be processed is any one of the data segments to be processed; if the current position of the indication cursor does not exceed the position of the last data to be processed in the target data segment to be processed, determining a compression mode according to the compression condition met by the current position of the indication cursor; and compressing the target data segment to be processed according to the determined compression mode.
As a possible implementation, the method further comprises: and if the current indication of the indication cursor is the data to be processed which is arranged at the forefront in the target data segment to be processed, adding a starting frame mark at the first position of the target compressed data segment corresponding to the target data segment to be processed in the compressed data array.
As a possible implementation manner, the compression mode flag includes a second-order difference flag, the compression mode includes a second-order difference compression, and the specific implementation manner of compressing the target data segment to be processed according to the determined compression mode may be: judging whether the current position of the indicating cursor meets a compression condition corresponding to the second-order differential compression; if yes, performing second-order differential compression on the data indicating the current position of the cursor and the frame length/second-order frame length behind the current position; judging whether a compression mode mark arranged at the last in the target compression data segment is a second-order difference mark or not; if the second-order difference mark exists, adding the data after the second-order difference compression at the next position of the target compressed data segment; if the target compressed data segment is not the second-order differential mark, adding the second-order differential mark at the next position of the target compressed data segment, and adding the data after the second-order differential compression at the next position of the second-order differential mark; the cursor will be instructed to move back by the frame length/second order frame length positions.
As a possible implementation manner, the compression mode flag further includes a first order difference flag, and the compression mode further includes a first order difference compression if the current position of the cursor does not satisfy the compression condition corresponding to the second order difference compression, where the method further includes: judging whether the current position of the indicating cursor meets a compression condition corresponding to first-order differential compression; if yes, performing first-order differential compression on the data indicating the current position of the cursor and the frame length/first-order frame length behind the current position; judging whether a compression mode mark arranged at the last in the target compression data section is a first-order difference mark or not; if the first-order difference mark exists, adding the data after the first-order difference compression at the next position of the target compressed data section; if the first-order difference mark is not the first-order difference mark, adding the first-order difference mark at the next position of the target compressed data section, and adding the data after the first-order difference compression at the next position of the first-order difference mark; the cursor will be indicated to move back by the frame length/first order frame length position.
As a possible implementation manner, the compression mode flag further includes a key frame flag, and if the current position of the pointing cursor does not satisfy the compression condition corresponding to the first-order difference compression, the method further includes: converting the data to be processed currently indicated by the indication cursor into binary data; judging whether a compression mode mark arranged at the last in the target compression data segment is a key frame mark or not; if the key frame mark exists, adding data in a binary form at the next position of the target compressed data segment; if the target compressed data segment is not the key frame mark, adding the key frame mark at the next position of the target compressed data segment, and adding the binary form data at the next position of the key frame mark; the pointing cursor is moved back by one position.
As one possible embodiment, the compression condition includes a data length condition, a compression feasibility condition, and a collision avoidance condition; the data length conditions of the second order differential compression are as follows: at least two pieces of compressed data are arranged in front of the current position of the indication cursor, and at least frame length/second-order frame length data are arranged at the current position and the back of the indication cursor; the compression feasibility conditions of the second-order differential compression are as follows: indicating the current position of the cursor and the second-order differential value of the frame length/second-order frame length data behind the cursor are smaller than the maximum value of the second-order frame, wherein the maximum value of the second-order frame is the maximum value represented by binary number of the second-order frame length; the collision avoidance conditions for the second order differential compression are: the data obtained after the second-order differential compression is different from the control frame mark; the data length condition of the first order difference compression is as follows: indicating at least one piece of compressed data in front of the current position of the cursor, and indicating at least one piece of data of frame length/first-order frame length behind the current position of the cursor; the compression feasibility conditions for the first order differential compression are: indicating the current position of the cursor and the first-order differential value of the frame length/first-order frame length data behind the cursor are all smaller than the maximum value of the first-order frame, and the maximum value of the first-order frame is the maximum value represented by binary number of the first-order frame length; the collision avoidance conditions for first order differential compression are: the data obtained after the first order difference compression is different from the control frame mark.
As a possible implementation, the control frame marker includes a start frame marker, a compression mode marker, and an end frame marker, the compressed data array includes at least one compressed data segment, and the method further includes: determining a data segment to be decompressed according to the compression parameters, wherein the data segment to be decompressed is any compressed data segment in which a first frame in a compressed data array is a starting frame mark and a last frame is an ending frame mark; dividing the data segment to be decompressed into a plurality of sub data segments to be decompressed according to the compression mode marks included in the data segment to be decompressed, wherein each sub data segment to be decompressed comprises the respective compression mode mark and the data to be decompressed; and respectively decompressing the plurality of sub data segments to be decompressed according to the respective compression mode marks.
As a possible implementation manner, the compression mode flag includes a key frame flag, a first order difference flag and a second order difference flag, and the specific implementation manner of respectively decompressing the multiple sub-data segments to be decompressed according to the respective compression mode flags may be: if the compression mode is marked as a key frame mark, decompressing the data to be decompressed included in the sub-data segment to be decompressed by utilizing a mode of converting the binary form into the decimal form; if the compression mode is marked as a first-order difference mark, decompressing the data to be decompressed included in the sub data segment to be decompressed according to a decompression mode corresponding to the first-order difference compression; if the compression mode mark is a second-order difference mark, decompressing the data to be decompressed included in the sub data segment to be decompressed according to a decompression mode corresponding to the second-order difference compression; and offsetting the data obtained after decompression according to the offset included by the compression parameters to obtain the corresponding data segment to be processed before the data segment to be decompressed is compressed.
As a possible implementation, the method further comprises: and recombining the to-be-processed data segments obtained by decompressing all the compressed data segments included in the compressed data array according to the time sequence to obtain a to-be-processed data set.
A second aspect of the embodiments of the present invention provides a data processing apparatus, including:
the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring compression parameters aiming at a data set to be processed, and the compression parameters comprise an offset, a maximum compression length and a control frame mark.
And the offset unit is used for offsetting the data to be processed included in the data set to be processed according to the offset.
And the dividing unit is used for dividing the shifted data to be processed into a plurality of data segments to be processed according to the maximum compression length.
And the processing unit is used for respectively compressing the plurality of data segments to be processed, and respectively adding the compressed data into the compressed data arrays according to the time sequence by utilizing the control frame marks corresponding to the compression mode adopted in the compression process.
A third aspect of the embodiments of the present invention provides another data processing apparatus, including:
and the acquisition unit is used for acquiring the value range of the data to be processed included in the data set to be processed.
And the determining unit is used for determining the offset, the frame length, the first-order frame length, the second-order frame length and the control frame mark aiming at the data set to be processed according to the value range of the data to be processed included in the data set to be processed. Wherein the start frame marker is used for marking the start of a compressed data segment in the compressed data array, and the start frame marker is located at the first position of the compressed data segment; the end frame marker is used for marking the end of one compressed data segment in the compressed data array, and the end frame marker is positioned at the last position of the compressed data segment.
And the dividing unit is used for offsetting the data to be processed included in the data set to be processed according to the offset and dividing the offset data to be processed into a plurality of data segments to be processed according to the maximum compression length.
The determining unit is further configured to determine, for a target to-be-processed data segment of the multiple data segments, a current position of an indication cursor in the target to-be-processed data segment, where the indication cursor is used to indicate next to-be-processed data, and the target to-be-processed data segment is any one of the multiple to-be-processed data segments.
And the processing unit is used for adding a starting frame mark at the first position of the target compressed data segment corresponding to the target data segment to be processed in the compressed data array if the current indication of the indication cursor is the data to be processed which is arranged at the top in the target data segment to be processed.
And the judging unit is used for judging whether the current position of the indication cursor meets the compression condition corresponding to the second-order differential compression or not if the current position of the indication cursor does not exceed the position of the last to-be-processed data arranged in the target to-be-processed data segment.
And the compression unit is used for performing second-order differential compression on the current position of the indication cursor and the following frame length/second-order frame length data if the current position of the indication cursor meets the compression condition corresponding to the second-order differential compression.
The judging unit is further configured to judge whether a compression mode flag arranged at the last in the target compressed data segment is a second-order difference flag.
The processing unit is further configured to add the second-order differentially compressed data at a next position of the target compressed data segment if the compression mode flag arranged at the last in the target compressed data segment is the second-order differential flag.
The processing unit is further configured to add a second order difference flag at a next position of the target compressed data segment and add the second order differentially compressed data at a next position of the second order difference flag if the last compression mode flag arranged in the target compressed data segment is not the second order difference flag.
And the moving unit is used for moving the indication cursor backwards by the frame length/second-order frame length.
As a possible embodiment, the determining unit in the data processing apparatus is further configured to determine whether a position where the pointing cursor is currently located satisfies a compression condition corresponding to first-order difference compression.
The compressing unit is further configured to perform first-order differential compression on the current position of the indication cursor and the following frame length/first-order frame length data if the current position of the indication cursor meets a compression condition corresponding to the first-order differential compression.
The judging unit is further configured to judge whether a compression mode flag arranged at the last in the target compressed data segment is a first-order difference flag.
The processing unit is further configured to add the first-order differential compressed data at a next position of the target compressed data segment if the last compression mode flag in the target compressed data segment is the first-order differential flag.
The processing unit is further configured to add a first-order difference flag at a next position of the target compressed data segment and add data after first-order difference compression at a next position of the first-order difference flag if the last compression mode flag arranged in the target compressed data segment is not the first-order difference flag.
The moving unit is also used for moving the indication cursor backwards by the frame length/a first-order frame length.
As a possible implementation manner, the processing unit in the data processing apparatus is further configured to convert the data to be processed currently indicated by the indication cursor into binary data if the current position of the indication cursor does not satisfy the compression condition corresponding to the first-order difference compression.
The judging unit is further configured to judge whether a compression mode flag arranged at the last in the target compressed data segment is a key frame flag.
The processing unit is further configured to add data in binary form at a next position of the target compressed data segment if the last compressed mode flag in the target compressed data segment is a key frame flag.
The processing unit is further configured to add a key frame mark at a next position of the target compressed data segment and add data in binary form at the next position of the key frame mark if the last compression mode mark in the target compressed data segment is not the key frame mark.
The moving unit is also used for moving the indication cursor backwards by one position.
As a possible implementation manner, the determining unit in the data processing apparatus is further configured to determine, according to the compression parameter, a data segment to be decompressed, where the data segment to be decompressed is any compressed data segment in which a first frame in the compressed data array is a start frame marker and a last frame is an end frame marker.
The dividing unit is further configured to divide the data segment to be decompressed into a plurality of sub-data segments to be decompressed according to the compression mode flag included in the data segment to be decompressed, where each sub-data segment to be decompressed includes the respective compression mode flag and data to be decompressed.
As a possible implementation, the data processing apparatus further includes:
and the decompression unit is used for respectively decompressing the plurality of sub data segments to be decompressed according to the respective compression mode marks.
As a possible implementation, the decompression unit is specifically configured to: if the compression mode is marked as a key frame mark, decompressing the data to be decompressed included in the sub-data segment to be decompressed by utilizing a mode of converting the binary form into the decimal form; if the compression mode is marked as a first-order difference mark, decompressing the data to be decompressed included in the sub data segment to be decompressed according to a decompression mode corresponding to the first-order difference compression; and if the compression mode mark is a second-order difference mark, decompressing the data to be decompressed included in the sub-data segment to be decompressed according to a decompression mode corresponding to the second-order difference compression.
As a possible implementation, the data processing apparatus further includes:
and the offset unit is used for offsetting the data obtained after decompression according to the offset included by the compression parameters to obtain the corresponding data segment to be processed before the data segment to be decompressed is compressed.
The processing unit is further configured to recombine the to-be-processed data segments obtained by decompressing all the compressed data segments included in the compressed data array according to the time sequence to obtain a to-be-processed data set.
A fourth aspect of the embodiments of the present invention provides a data processing apparatus, including: the method comprises a processor, a communication interface and a memory, wherein the processor, the communication interface and the memory are connected with each other, the memory is used for storing a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method provided by the first aspect of the embodiment of the present invention.
A fifth aspect of embodiments of the present invention provides a computer-readable storage medium, in which a computer program is stored, the computer program comprising program instructions that, when executed by a processor, cause the processor to perform the method provided by the first aspect of embodiments of the present invention.
The embodiment of the invention has the following beneficial effects: offsetting the data to be processed through the offset in the obtained compression parameters, and dividing the data to be processed into a plurality of data sections to be processed according to the maximum compression length in the compression parameters; and respectively compressing the plurality of data segments to be processed, and respectively adding the compressed data into the compressed data arrays according to the time sequence according to the control frame marks corresponding to the compression mode adopted in the compression process. The data compression method provided by the embodiment of the invention can realize higher compression rate with smaller calculation amount.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic flow chart diagram of a data processing method provided by an embodiment of the invention;
FIG. 2 is a diagram illustrating a data format of a compressed data segment in an array of compressed data according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart diagram of another data processing method provided by an embodiment of the invention;
FIG. 4 is a diagram illustrating binary conversion of differential data according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of decimal conversion to binary in an embodiment of the present invention;
FIG. 6 is a schematic flow chart of decompressing compressed data in an embodiment of the present invention;
FIG. 7 is a schematic diagram of the compression and decompression process of two-dimensional data in an embodiment of the invention;
FIG. 8 is a schematic block diagram of a data processing apparatus provided by an embodiment of the present invention;
FIG. 9 is a schematic block diagram of another data processing apparatus provided by an embodiment of the present invention;
fig. 10 is a schematic block diagram of another data processing apparatus provided in an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, which is a schematic flow chart of a data processing method according to an embodiment of the present invention, the method shown in fig. 1 may include:
101. and acquiring compression parameters aiming at the data set to be processed, wherein the compression parameters comprise an offset, a maximum compression length and a control frame mark.
Specifically, the data processing apparatus acquires compression parameters for a set of data to be processed. The data set to be processed is formed by arranging and combining a plurality of data to be processed according to a time sequence. The compression parameters of the data set to be processed are used for determining the structure of the data processing device, and the structure of the data processing device is some parameter indexes in software. It will be appreciated that the data processing apparatus may be a terminal or a server.
The offset in the compression parameter is used for ensuring that each data in the data to be processed is a positive value from zero so as to facilitate subsequent processing; the maximum compression length in the compression parameters is a fixed parameter of the data processing device and is used for representing the maximum data volume which can be compressed by the data processing device at one time; the control frame mark is used for marking the data expression form behind the current position, and the control frame mark does not carry data information. The control frame flag may specifically include: a start frame marker, a compression mode marker, and an end frame marker, wherein the compression mode marker includes a first order difference marker, a second order difference marker, and a key frame marker.
Illustratively, the start frame marker is used to indicate the start of a compressed data segment in the compressed data array, and the end frame marker is used to indicate the end of a compressed data segment in the compressed data array; the first-order difference mark is used for indicating that the data after the current position is the data compressed by adopting the first-order difference, and the second-order difference mark is used for indicating that the data after the current position is the data compressed by adopting the second-order difference; and the key frame flag is used to indicate that the data after the current position is data that cannot be compressed.
For example, as shown in fig. 2, fig. 2 shows a data format of a compressed data segment in the compressed data array, data after the compression mode is marked as the key frame data, data after the compression mode is marked as the first-order difference data, and data after the compression mode is marked as the second-order difference data; the start frame marker and the end frame marker mark the beginning and the end, respectively, of a compressed data segment in a compressed data array.
102. And offsetting the data to be processed included in the data set to be processed according to the offset, and dividing the offset data to be processed into a plurality of data segments to be processed according to the maximum compression length.
Specifically, the data processing apparatus adds an offset to the to-be-processed data included in the to-be-processed data set, and divides the offset to-be-processed data into a plurality of to-be-processed data segments according to the maximum compression length. In specific implementation, the data processing device takes the inverse number of the minimum value in the to-be-processed data included in the to-be-processed data set as the offset of the to-be-processed data, and the data processing device performs offset on the to-be-processed data according to the offset. Illustratively, if the data range in the set of data to be processed is-100 to 200, the offset of the data to be processed is 100. The data processing device performs the offset on the data to be processed according to the offset in the following way: and adding 100 to each data in the data set to be processed to obtain the data range of the data to be processed from 0 to 300.
The maximum compression length is an inherent parameter of the data processing device, and each data processing device corresponds to one maximum compression length. The number of the to-be-processed data segments obtained by dividing the to-be-processed data by the data processing device is a numerical value obtained by rounding up (the length/maximum compression length of the to-be-processed data set). Illustratively, if the data volume of the data to be processed included in the data set to be processed is 100M, and the maximum compression length of the data processing apparatus is 26M, the data set to be processed is divided into 4(100/26 rounding up) data segments according to the maximum compression length.
Alternatively, the data in the data set to be processed may be multi-dimensional data, such as two-dimensional point coordinates (x, y) or three-dimensional dataThe accelerometer (x, y, z) of (a), after the data processing apparatus divides the shifted data to be processed into a plurality of data segments to be processed according to the maximum compression length, the method may further include: and separating data of each dimension according to the dimension of the target data segment to be processed aiming at the target data segment to be processed in the plurality of data segments to be processed, and storing the data of each dimension separately according to the time sequence. For example, if the data set to be processed is a two-dimensional point coordinate data set, the data in the target data segment to be processed can be expressed as { (x)1，y1)，(x2，y2)，(x3，y3)，(x4，y4)，…(xn，yn) Dividing the data into subsets { x ] according to the dimension of the data to be processed1,x2,x3,x4,…,xnAnd { y }1,y2,y3,y4,…,ynAnd arranging the data in each subset according to a time sequence.
It can be understood that, if the data in the data set to be processed is multidimensional data, the compression parameters corresponding to the dimensions need to be determined according to the value range of the data of the dimensions. In order to achieve higher compression efficiency, for data of different dimensions, if the value ranges are different greatly, data processing devices with different compression parameters need to be adopted for processing. Illustratively, if the data to be processed is a two-dimensional point (x)i，yi) Data set of coordinates, and xiHas a value in the range of-20 to 350, yiThe value range of (1) is-200 to 1000; x is theniIs 20 and yiThe offset of (2) is 200, the difference between the two is ten times, if the same parameters are adopted, the x can not be simultaneously realizediAnd yiThe best compression efficiency.
103. And respectively compressing the plurality of data segments to be processed, and respectively adding the compressed data into the compressed data arrays according to the time sequence by using the control frame marks corresponding to the compression mode adopted in the compression process.
Specifically, the data processing device respectively compresses a plurality of data segments to be processed, and adds the compressed data into the compressed data arrays according to the time sequence by using the control frame marks corresponding to the compression mode adopted in the compression process. The compressed data array is used for storing the compressed data in sequence, and the compressed data array also comprises a control frame mark besides storing the compressed data. The control frame mark is used for marking the compression mode corresponding to the data after the control frame mark. The control frame marker includes: a start frame marker, a compression mode marker, and an end frame marker, wherein the compression mode marker includes a first order difference marker, a second order difference marker, and a key frame marker.
It is understood that the data processing apparatus may perform the compression process on a plurality of data segments to be processed in parallel, but the data generated after the compression needs to be arranged in time sequence. Specifically, after the data to be processed included in the data set to be processed is shifted, the data set to be processed is divided into a plurality of data segments to be processed; the data of the data segments to be processed are arranged in time sequence and sent to a data processing device for compression; the compressed data is still arranged in chronological order. And if the data to be processed included in the data set to be processed is multidimensional data, arranging the data of each dimensionality in each compressed data segment according to the time sequence.
By implementing the embodiment of the invention, the data processing device offsets the data to be processed according to the offset in the acquired compression parameters, and divides the data to be processed into a plurality of data segments to be processed according to the maximum compression length in the compression parameters; and respectively compressing the plurality of data segments to be processed, and respectively adding the compressed data into the compressed data arrays according to the time sequence according to the control frame marks corresponding to the compression mode adopted in the compression process. The above embodiment divides the data in the data set to be processed into data segments and compresses the divided data segments to be processed, so that a higher compression rate can be achieved with a smaller amount of calculation.
Referring to fig. 3, fig. 3 is a schematic flow chart of another data processing method according to an embodiment of the present invention, where the method includes:
301. and acquiring the value range of the data to be processed included in the data set to be processed.
Specifically, the data processing device obtains a value range of the data set to be processed.
It can be understood that the value range of the data set to be processed is determined by the maximum value and the minimum value in the data to be processed. If the data to be processed included in the data set to be processed is multidimensional data, the value ranges of the dimensional data in the data set to be processed need to be determined and stored respectively.
302. And determining the offset, the frame length, the first-order frame length, the second-order frame length and the control frame mark aiming at the data set to be processed according to the value range of the data to be processed included in the data set to be processed.
Wherein the control frame mark comprises a start frame mark, a compression mode mark and an end frame mark. Wherein the start frame marker is used for marking the start of a compressed data segment in the compressed data array, and the start frame marker is located at the first position of the compressed data segment; the end frame marker is used to mark the end of a compressed data segment in the compressed data array, the end frame marker being located at the last position of the compressed data segment. The data processing device determines the offset, the frame length, the first-order frame length, the second-order frame length and the control frame mark of the data set to be processed according to the value range of the data to be processed included in the data set to be processed.
In a specific implementation, if the data to be processed included in the data set to be processed is multidimensional data, taking the opposite number of each dimension minimum value in the data set to be processed as the offset of the corresponding dimension data; for specific data of a certain dimension, counting the number obtained by subtracting the minimum value from the maximum value and adding 5 as an intermediate number, and calculating the minimum byte number required by the intermediate number, wherein the byte number is the frame length; taking the maximum 5 numbers which can be represented by the frame length as control frame marks which respectively represent a starting frame mark, a first-order difference mark, a second-order difference mark, a key frame mark and an ending frame mark; if the frame length is odd number, taking (frame length-1)/2 as first order frame length, (frame length-1)/4 as second order frame length; if the frame length is an even number, taking (frame length)/2 as a first-order frame length and (frame length)/4 as a second-order frame length; and taking the maximum value represented by the binary number of the first-order frame length as the maximum value of the first-order frame, and taking the maximum value represented by the binary number of the second-order frame length as the maximum value of the second-order frame.
303. And offsetting the data to be processed included in the data set to be processed according to the offset, and dividing the offset data to be processed into a plurality of data segments to be processed according to the maximum compression length.
Specifically, the data processing apparatus shifts the to-be-processed data included in the to-be-processed data set according to the offset determined in step 302, and divides the to-be-processed data after the offset into a plurality of to-be-processed data segments according to the maximum compression length. It will be appreciated that the length of the plurality of data segments into which the data processing apparatus divides by the maximum compression length may be different.
Illustratively, if the data set to be processed is a decimal value sequence with a length of 1024 and the maximum compression length of the data processing device is 100 decimal numbers, 11 data segments to be processed are obtained by dividing according to the maximum compression length, wherein each data segment to be processed in the first 10 data segments to be processed has 100 data, and the 11 th data segment to be processed has only 24 data.
304. And determining the current position of the indicating cursor in the target data segment to be processed aiming at the target data segment to be processed in the plurality of data segments to be processed.
Specifically, the indication cursor is used for indicating the next data to be processed, and the target data segment to be processed is any one of the multiple data segments to be processed. The data processing device determines the current position of the indicating cursor in the target data segment to be processed aiming at the target data segment to be processed in the plurality of data segments to be processed.
305. And if the current indication of the indication cursor is the data to be processed which is arranged at the forefront in the target data segment to be processed, adding a starting frame mark at the first position of the target compressed data segment corresponding to the target data segment to be processed in the compressed data array.
Specifically, the data processing apparatus adds a start frame marker at a first position of a target compressed data segment corresponding to a target to-be-processed data segment in the compressed data array. The data processing device determines the frame length according to the value range of the data to be processed included in the data set to be processed, then uses the maximum 5-bit number which can be represented by the frame length as a control frame mark, determines the numerical value of the start frame mark, and adds the start frame mark to the first position of the target compressed data segment in the compressed data array.
For example, if the calculated frame length is 2 bytes, then 2 bytes can represent the maximum number as shown in table 1, and if 65535 is used as the start frame marker, 65534 is used as the key frame marker, 65533 is used as the second-order frame marker, 65532 is used as the first-order frame marker, and 65531 is used as the end frame marker, then the start frame marker 1111111111111111 is added to the first position of the target compressed data segment in the compressed data array.
TABLE 1
Maximum 5-bit number expressed by 2 bytes Corresponding binary representation
65535 11111111 11111111
65534 11111111 11111110
65533 11111111 11111101
65532 11111111 11111100
65531 11111111 11111011
It should be further noted that, in the embodiment of the present invention, the multi-byte data converted into the binary system may be added to the target compressed data segment of the compressed data array in a big-end mode (the high byte of the data is stored in the low address of the memory, and the low byte of the data is stored in the high address of the memory) or a small-end mode (the low byte of the data is stored in the low address of the memory, and the high byte of the data is stored in the high address of the memory). But the same mode needs to be uniformly adopted in the whole compression and decompression system.
306. And if the current position of the indication cursor does not exceed the position of the last to-be-processed data arranged in the target to-be-processed data segment, judging whether the current position of the indication cursor meets the compression condition corresponding to the second-order differential compression.
Specifically, the compression method of the data to be processed includes first-order differential compression and second-order differential compression. It is understood that performing the first order differential compression or the second order differential compression requires satisfying the corresponding compression conditions, wherein the compression conditions include a data length condition, a compression feasibility condition, and a collision avoidance condition. The data length condition for performing the second-order differential compression is as follows: at least two pieces of compressed data are arranged in front of the current position of the indication cursor, and at least frame length/second-order frame length data are arranged at the current position and the back of the indication cursor; the compression feasibility conditions for performing the second-order differential compression are as follows: indicating the current position of the cursor and the second-order differential value of the frame length/second-order frame length data behind the cursor are smaller than the maximum value of the second-order frame, wherein the maximum value of the second-order frame is the maximum value which can be expressed by binary number of the second-order frame length; the collision avoidance conditions for performing the second order differential compression are: the data obtained after the second order differential compression is different from the control frame mark.
Specifically, the second-order differential compression data length condition is to determine that the data currently indicated by the indication cursor can be subjected to second-order differential operation, and a value obtained after the second-order differential operation just generates one frame of data. For example, if the pointing cursor currently indicates the first data (i.e. 15) in the target to-be-processed data segment 15,17,19,21,18 …, then according to the calculation method of the second order difference: (data [ i ] -data [ i-1]) - (data [ i-1] -data [ i-2]), it can be seen that data [ i ] is 15, but the values of data [ i-1] and data [ i-2] cannot be determined, and therefore the corresponding second order difference value cannot be calculated.
Specifically, the compression feasibility condition of the second-order differential compression is to ensure that the value obtained by performing the second-order differential operation can be represented by a binary number of the second-order frame length. Illustratively, if the frame length is 1 byte, the second order frame length is 1/2 bytes, i.e., a 4-bit binary number. The maximum value that the second-order frame length can express is 15, and if the second-order differential values indicating the current position of the cursor and the following 2 digits (obtained by frame length/second-order frame length calculation) are both less than 15, the compression feasibility condition of the second-order differential compression is met.
Specifically, the collision avoidance condition of the second order differential compression is for ensuring the uniqueness of the control frame. As previously described, the role of the control frame is to mark the representation of the data after the current position. If the uniqueness of the control frame in the compressed data array cannot be ensured, decompression errors occur when data is decompressed.
It can be understood that the data to be processed arranged at the last in the target data segment to be processed is the last data to be processed in the target data segment to be processed. And if the indicating cursor currently indicates the last to-be-processed data in the target to-be-processed data segment and the currently indicated data is completely compressed, ending the compression of the target to-be-processed data segment, and adding an ending frame mark at the next position in the target compressed data segment.
307. And if the current position of the indicating cursor meets the compression condition corresponding to the second-order differential compression, performing second-order differential compression on the current position of the indicating cursor and the frame length/second-order frame length data behind the current position of the indicating cursor.
Specifically, a second-order differential value of the position of the indicating cursor and the frame length/second-order frame length data behind the indicating cursor is calculated, and the calculated numerical values are converted into binary numbers which are sequentially arranged according to a time sequence. Each 8 bits are grouped into one byte, which constitutes one frame. The second-order difference calculation method comprises the following steps: (data [ i ] -data [ i-1]) - (data [ i-1] -data [ i-2]), wherein i is the position where the cursor is currently located, data is the target data segment to be processed, and data [ i ] represents the ith data in the target data segment to be processed.
308. And judging whether the compression mode mark arranged at the last in the target compression data segment is a second-order difference mark or not.
Specifically, the compression mode flag arranged at the last in the target compressed data segment is the latest compression mode flag added in the target compressed data segment. It is to be understood that the compressed data corresponding to the compression mode flag is also included in the target compressed data segment, which is arranged after the last compression mode flag. For example, if the compression mode flag arranged at the last in the compressed data segment is a key frame flag, the compression mode flag further includes key frame data after the compression mode flag; if the compression mode flag arranged at the last in the target compressed data segment is a first-order differential compression flag, the compression mode flag is followed by first-order differential compressed data.
309. And if the last compression mode mark in the target compressed data segment is a second-order differential mark, adding the data after second-order differential compression at the next position of the target compressed data segment.
It can be understood that, if the compression mode flag arranged at the last in the target compressed data segment is the second-order differential flag, it indicates that the data after the second-order differential flag in the target compressed data segment is the data obtained after the last second-order differential compression. Under the condition, the data after the second-order differential compression is directly added at the next position of the target compressed data segment, and the implementation mode can ensure that a plurality of second-order differential compressed data only need one second-order differential mark to be identified in the target compressed data segment, so that the compression efficiency of the data can be improved.
310. And if the last compression mode mark arranged in the target compressed data segment is not the second-order difference mark, adding the second-order difference mark at the next position of the target compressed data segment, and adding the data after the second-order difference compression at the next position of the second-order difference mark.
It can be understood that, if the compression mode flag arranged at the last in the target compressed data segment is not the second-order differential flag, indicating that the last compression mode in the target compressed data segment is not the second-order differential compression, it is necessary to add the second-order differential flag in the target compressed data segment first, and then add the data after the second-order differential compression at the next position of the second-order differential flag.
Specifically, the data added to the target compressed data segment is in a binary form, that is, a value obtained by calculating the second-order difference needs to be converted into a binary form, and then the binary form is added to the target compressed data segment. For example, if the frame length is 3 bytes, when the compression condition of the second-order differential compression is satisfied, each frame needs to include 6 second-order differential data, and if the second-order differential operation obtains a value: 14,13,14,13,14,13, the process of binary converting the 6 second-order differential data is shown in fig. 4.
311. The cursor will be instructed to move back by the frame length/second order frame length positions.
Further, if it is determined that the current position of the pointing cursor does not satisfy the compression condition corresponding to the second-order differential compression, it may be determined whether the position of the pointing cursor satisfies the compression condition corresponding to the first-order differential compression, specifically including the following steps 312 to 317.
312. And judging whether the current position of the indicating cursor meets the compression condition corresponding to the first-order difference compression.
The compression conditions corresponding to the first-order differential compression comprise a data length condition, a compression feasibility condition and a collision avoidance condition. The data length condition of the first-order difference compression is as follows: indicating at least one piece of compressed data in front of the current position of the cursor, and indicating at least one piece of data of frame length/first-order frame length behind the current position of the cursor; the compression feasibility conditions for the first order differential compression are: indicating the current position of the cursor and the first-order difference value of the frame length/first-order frame length data behind the cursor are all smaller than the maximum value of the first-order frame, and the maximum value of the first-order frame is the maximum value represented by binary number of the first-order frame length; the collision avoidance conditions for first order differential compression are: the data obtained after the first order difference compression is different from the control frame mark.
Specifically, the data length condition of the first-order difference compression is used for ensuring that the data at the current position can be subjected to the first-order difference operation, and just one frame of data is generated after the first-order difference operation is performed. The compression feasibility condition of the first order difference compression is used to ensure that the numerical value obtained by the first order difference operation can be represented by a binary number of the length of the first order frame. The collision avoidance condition of the first order differential compression is used to ensure the uniqueness of the control frame.
313. And if the current position of the indicating cursor meets the compression condition corresponding to the first-order differential compression, performing the first-order differential compression on the current position of the indicating cursor and the frame length/first-order frame length data behind the current position of the indicating cursor.
Specifically, if the current position of the pointing cursor meets the data length condition, the compression feasibility condition and the collision avoidance condition corresponding to the first-order difference compression at the same time, the data processing device performs the first-order difference compression on the current position of the pointing cursor and the following frame length/first-order frame length data. The first-order difference calculation method comprises the steps of data [ i ] -data [ i-1], wherein i indicates the current position of the cursor, data is a target data segment to be processed, and data [ i ] represents ith data in the target data segment to be processed.
314. And judging whether the compression mode mark arranged at the last in the target compression data section is a first-order difference mark.
It is to be understood that the compression mode flag arranged last in the target compressed data segment is the compression mode flag newly added in the target compressed data segment. Compressed data corresponding to the compression mode flag is included in the target compressed data segment after the last compression mode flag. For example, if the compression mode flag arranged at the end in the target compressed data segment is a second-order differential flag, the second-order differential data is included after the second-order differential flag in the target compressed data segment; if the last compression mode flag in the target compressed data segment is a key frame flag, the key frame data in binary form is included after the key frame flag in the target compressed data segment.
315. And if the last compression mode mark in the target compressed data segment is a first-order difference mark, adding the data after the first-order difference compression at the next position of the target compressed data segment.
It can be understood that, if the last compression mode flag in the target compressed data segment is the first-order difference flag, indicating that the last compression is the first-order difference compression, the data after the first-order difference compression is directly added to the target compressed data segment.
316. And if the last compression mode mark in the target compressed data segment is not the first-order difference mark, adding the first-order difference mark at the next position of the target compressed data segment, and adding the data after the first-order difference compression at the next position of the first-order difference mark.
It can be understood that if the compression mode flag arranged at the last in the target compressed data segment is not the first-order difference flag, it indicates that the last data compression is not the first-order difference compression, and therefore, it is necessary to add the first-order difference flag in the target compressed data segment and then add the data after the first-order difference compression at the next position of the first-order difference flag.
317. The cursor will be indicated to move back by the frame length/first order frame length position.
Further, if it is determined that the current position of the pointing cursor does not satisfy the compression condition corresponding to the first-order difference compression, the method may further include the following steps 318-322.
318. And if the current position of the indicating cursor does not meet the compression condition corresponding to the first-order difference compression, converting the data to be processed, which is currently indicated by the indicating cursor, into binary data.
Specifically, the data processing device converts the data to be processed currently indicated by the indication cursor into binary data, i.e., converts a decimal number into a binary number representation. Illustratively, as shown in FIG. 5, a decimal number 12345 is converted to a 2-byte binary number. The position of the indication cursor may not satisfy the second order difference condition nor the first order difference condition, and at this time, data in the target data segment to be processed is not subjected to differential compression, but is converted as key frame data.
319. And judging whether the compression mode mark arranged at the last in the target compression data segment is a key frame mark.
It is understood that the first position in the target compressed data segment is the start frame marker, and the data after the start frame marker is the key frame data according to the compression conditions of the first order differential compression and the second order differential compression. In the embodiment of the invention, if the compression mode mark is not found in the target compressed data segment, a key frame mark is added at the next position of the start frame mark, and then the judgment of the compression mode mark is carried out.
320. If the last compression mode flag in the target compressed data segment is a key frame flag, then data in binary form is added at the next position of the target compressed data segment.
It is understood that the last key frame mark in the target compressed data segment may be the key frame mark added at the next position of the start frame mark in step 319, or the key frame mark added during the last compression of the target compressed data segment.
321. If the last compression mode flag in the target compressed data segment is not a key frame flag, adding a key frame flag at the next position of the target compressed data segment and adding data in binary form at the next position of the key frame flag.
Specifically, if the last compression mode flag in the target compressed data segment is not a key frame flag, it indicates that the last compression is a second-order differential compression or a first-order differential compression, and a key frame flag needs to be added at the next position in the target compressed data segment, and then binary data needs to be added.
322. The pointing cursor is moved back by one position.
By the embodiment of the invention, the corresponding compression modes, such as first-order differential compression and second-order differential compression, can be determined according to the compression condition met by the current position of the indicating cursor. If the position of the indication cursor meets the second-order differential compression, performing the second-order differential compression; when the position of the indication cursor does not meet the compression condition corresponding to the second-order differential compression, whether the current position of the indication cursor meets the compression condition corresponding to the first-order differential compression is continuously judged; and if the position of the indicating cursor does not meet the compression condition corresponding to the first-order difference compression, storing the data to be processed indicated by the indicating cursor in the form of key frame data. Wherein the compression rate of the second-order differential compression is higher than that of the first-order differential compression. It can be seen that, in the above manner, it is possible to achieve a greater compression rate with a smaller amount of computation for the data to be processed included in the data set to be processed.
Optionally, the data processing method further includes a data decompression process, as shown in fig. 6, the data decompression process includes:
601. and determining the data segment to be decompressed according to the compression parameters.
The data segment to be decompressed is any compressed data segment in which the first frame in the compressed data array is a start frame marker, and the last frame is an end frame marker.
Specifically, the data in the compressed data array is read frame by frame in time sequence, and whether the data of each frame is a control frame mark, such as a start frame mark and an end frame mark, is determined. When a pair of start frame mark and end frame mark is found, it is determined that a target data segment to be processed is located between the start frame mark and the end frame mark. It can be understood that the specific implementation method for judging whether the data of each frame is the control frame mark may be to compare by bit, or to convert the data of each frame into a decimal value form, and to compare the decimal value with the control frame in the decimal representation form. The compressed data array comprises a plurality of data segments to be decompressed.
602. Dividing the data segment to be decompressed into a plurality of sub data segments to be decompressed according to the compression mode marks included in the data segment to be decompressed.
Wherein, each sub data segment to be decompressed comprises a respective compression mode flag and data to be decompressed.
Specifically, if the compression mode mark in the sub data segment to be decompressed is a key frame mark, determining that the data after the key frame mark is key frame data; if the compression mode mark in the sub data segment to be decompressed is a first-order difference mark, determining that the data after the first-order difference mark is the data after the first-order difference compression; and if the compression mode mark in the sub data segment to be decompressed is a second-order differential mark, determining that the data after the second-order differential mark is the data after the second-order differential compression. It is understood that the sub-data segment to be decompressed only contains one compressed mode flag, but the compressed mode flag contains one or more compressed data.
603. And respectively decompressing the plurality of sub data segments to be decompressed according to the respective compression mode marks.
Specifically, if the compression mode is marked as a key frame mark, the data to be decompressed included in the sub-data segment to be decompressed is decompressed by converting the binary form into the decimal form.
And if the compression mode is marked as a first-order difference mark, decompressing the data to be decompressed included in the sub-data section to be decompressed according to a decompression mode corresponding to the first-order difference compression. Specifically, a current frame in the sub data segment to be decompressed is sequentially divided into a plurality of data blocks by taking a first-order frame length as a unit; converting each data block into data in a decimal form according to the numerical value represented in the binary form, wherein the obtained numerical value is a first-order difference value calculated in the compression process; sequentially decompressing data according to a formula data [ i ] ═ data [ i-1] + L1[ i ], wherein data [ i ] represents the ith data to be decompressed, data [ i-1] is the last decompressed data, and L1[ i ] is the first-order difference corresponding to the ith decompressed data, and sequentially corresponds to the first-order difference converted from the current frame.
And if the compression mode mark is a second-order difference mark, decompressing the data to be decompressed included in the sub-data segment to be decompressed according to a decompression mode corresponding to the second-order difference compression. Specifically, the current frame in the sub data segment to be decompressed is divided into a plurality of data blocks in sequence by taking the length of the second-order frame as a unit; converting each data block into data in a decimal form according to the numerical value represented in the binary form, wherein the obtained numerical value is a second-order difference value calculated in the compression process; and sequentially decompressing data according to a formula data [ i ] +2 data [ i-1] -data [ i-2], wherein data [ i ] represents the ith data to be decompressed, data [ i-1] is the data decompressed last, data [ i-2] is the data decompressed last, and L2[ i ] is the second-order difference corresponding to the ith decompressed data and sequentially corresponds to the first-order difference converted from the current frame.
And offsetting the data obtained after decompression according to the offset included by the compression parameters to obtain the corresponding data segment to be processed before the data segment to be decompressed is compressed.
It will be appreciated that if decompression of all frames in the segment of data to be decompressed is not complete, decompression of the next frame of data continues until decompression of all frames is complete. Specifically, whether decompression of all frames in the data segment to be decompressed is completed can be determined by determining whether decompression of data before the end frame in the data segment to be decompressed has been completed. Further, if the decompression of all the data segments in the data to be decompressed is not completed, the next data segment to be decompressed is continuously acquired until the decompression process of all the data segments is completed. Specifically, whether to complete decompression of all data segments may be determined by determining whether a control frame flag, such as a start frame flag and an end frame flag, can be matched, or whether to complete decompression of all data segments may be determined by determining whether the number of decompressed data matches the number of data recorded during compression.
604. And recombining the to-be-processed data segments obtained by decompressing all the compressed data segments included in the compressed data array according to the time sequence to obtain a to-be-processed data set.
For better understanding of the embodiments of the present invention, an exemplary description of the compression and decompression processes in the embodiments of the present invention is provided below. As shown in fig. 7, taking the compression and decompression process of two-dimensional data as an example, firstly, in the compression process of data, the target number to be processed is to be processedAccording to the paragraph { (x)1，y1)，(x2，y2)，(x3，y3)，(x4，y4)，…(xn，yn) Separating data of each dimensionality in the subset to obtain a subset { x }1,x2,x3,x4,…,xnAnd { y }1,y2,y3,y4,...,yn}; then, the two data segments (data segment 1 and data segment 2) are compressed according to the compression parameters, and for a specific compression process, please refer to the method in the embodiment corresponding to fig. 1 or fig. 3, to obtain a compressed data array. In the process of decompressing data, firstly, a data segment to be decompressed is found according to the control frame flag, such as the start frame flag and the end frame flag, and the data in the data segment to be decompressed is decompressed according to the steps shown in fig. 6 in the above embodiment. Decompression results in a data set of two dimensions, i.e. { x }1,x2,x3,x4,…,xnAnd { y }1,y2,y3,y4,…,ynRecombining the original data to be processed into an original data set to be processed according to the time sequence: { (x)1，y1)，(x2，y2)，(x3，y3)，(x4，y4)，…(xn，yn)}。
Referring to fig. 8, fig. 8 is a schematic block diagram of a data processing apparatus according to an embodiment of the present invention, for implementing corresponding functions in the data processing apparatus in fig. 1, as shown in fig. 8, the data processing apparatus includes the following units:
an obtaining unit 801, configured to obtain compression parameters for a data set to be processed, where the compression parameters include an offset, a maximum compression length, and a control frame flag.
The shifting unit 802 is configured to shift the to-be-processed data included in the to-be-processed data set according to the shift amount.
A dividing unit 803, configured to divide the shifted data to be processed into multiple data segments to be processed according to the maximum compression length.
And the processing unit 804 is configured to compress the multiple data segments to be processed, and add the compressed data to the compressed data arrays according to a time sequence by using the control frame markers corresponding to the compression mode adopted in the compression process.
According to the embodiment of the present invention, the obtaining unit 801 obtains the compression parameters for the data set to be processed, and the shifting unit 802 shifts the data to be processed included in the data set to be processed according to the shift amount. The dividing unit 803 divides the shifted data to be processed into a plurality of data segments to be processed according to the maximum compression length. The processing unit 804 respectively compresses the plurality of data segments to be processed, and adds the compressed data into the compressed data arrays according to the time sequence by using the control frame marks corresponding to the compression mode adopted in the compression process. It can be seen that, the compression method for the data to be processed included in the data set to be processed can achieve higher compression efficiency with a smaller amount of calculation.
Referring to fig. 9, fig. 9 is a schematic block diagram of another data processing apparatus provided in an embodiment of the present invention, for implementing corresponding functions in the data processing apparatus in the embodiment of fig. 3, as shown in fig. 9, the data processing apparatus includes the following units:
an obtaining unit 901, configured to obtain a value range of to-be-processed data included in the to-be-processed data set.
The determining unit 902 is configured to determine, according to a value range of data to be processed included in the data set to be processed, an offset, a frame length, a first-order frame length, a second-order frame length, and a control frame flag for the data set to be processed.
The dividing unit 903 is configured to offset, according to the offset, data to be processed included in the data set to be processed, and divide the offset data to be processed into multiple data segments to be processed according to the maximum compression length.
The determining unit 902 is further configured to determine, for a target data segment to be processed among the plurality of data segments, a current position of the pointing cursor in the target data segment to be processed.
The processing unit 904 is configured to add a start frame marker to a first position of a target compressed data segment corresponding to a target to-be-processed data segment in the compressed data array if the current indication of the indication cursor is the data to be processed which is arranged at the top in the target to-be-processed data segment.
The determining unit 905 is configured to determine whether the current position of the indication cursor meets a compression condition corresponding to the second-order differential compression if the current position of the indication cursor does not exceed the position of the last to-be-processed data in the target to-be-processed data segment.
A compressing unit 906, configured to perform second order differential compression on the position where the cursor is located and the following frame length/second order frame length data if the position where the cursor is located meets the compression condition corresponding to the second order differential compression.
The determining unit 905 is further configured to determine whether a compression mode flag arranged at the last in the target compressed data segment is a second-order difference flag.
The processing unit 904 is further configured to add second-order differential compressed data to a next position of the target compressed data segment if the last compression mode flag in the target compressed data segment is a second-order differential flag.
The processing unit 904 is further configured to add a second order difference flag at a next position of the target compressed data segment and add second order difference compressed data at a next position of the second order difference flag if the last compression mode flag in the target compressed data segment is not the second order difference flag.
A moving unit 907 for moving the pointing cursor backward by a frame length/second order frame length position.
As a possible embodiment, the judging unit 905 in the data processing apparatus is further configured to judge whether the position where the indication cursor is currently located satisfies a compression condition corresponding to first-order difference compression.
The compressing unit 906 is further configured to, if the position where the cursor is currently located meets a compression condition corresponding to the first-order difference compression, perform the first-order difference compression on the data indicating the position where the cursor is currently located and the frame length/first-order frame length behind the position where the cursor is currently located.
The determining unit 905 is further configured to determine whether a compression mode flag arranged at the last in the target compressed data segment is a first-order difference flag.
The processing unit 904 is further configured to add the first-order difference compressed data at a next position of the target compressed data segment if the last compression mode flag in the target compressed data segment is the first-order difference flag.
The processing unit 904 is further configured to add a first-order difference flag at a next position of the target compressed data segment and add the first-order difference compressed data at a next position of the first-order difference flag if the last compression mode flag in the target compressed data segment is not the first-order difference flag.
The moving unit 907 is further configured to move the cursor backward by a frame length/first order frame length position.
As a possible implementation manner, the processing unit 904 in the data processing apparatus is further configured to convert the data to be processed currently indicated by the indication cursor into binary data if the current position of the indication cursor does not satisfy the compression condition corresponding to the first-order difference compression.
The determining unit 905 is further configured to determine whether a compression mode flag arranged at the last in the target compressed data segment is a key frame flag.
The processing unit 904 is further configured to add data in binary form at a next position of the target compressed data segment if the last compressed mode flag in the target compressed data segment is a key frame flag.
The processing unit 904 is further configured to add a key frame mark at a next position of the target compressed data segment and add data in binary form at the next position of the key frame mark if the last compression mode mark in the target compressed data segment is not the key frame mark.
The moving unit 907 is further configured to move the pointing cursor backward by one position.
As a possible implementation manner, the determining unit 902 in the data processing apparatus is further configured to determine, according to the compression parameter, a data segment to be decompressed, where the data segment to be decompressed is any compressed data segment in which a first frame in the compressed data array is a start frame marker and a last frame is an end frame marker.
The dividing unit 903 is further configured to divide the data segment to be decompressed into a plurality of sub data segments to be decompressed according to the compression mode flag included in the data segment to be decompressed, where each sub data segment to be decompressed includes the respective compression mode flag and data to be decompressed.
As a possible implementation, the data processing apparatus further includes:
the decompressing unit 908 is configured to decompress the plurality of sub-data segments to be decompressed according to the respective compression mode flags.
As a possible implementation manner, the decompression unit 908 is specifically configured to: if the compression mode is marked as a key frame mark, decompressing the data to be decompressed included in the sub-data segment to be decompressed by utilizing a mode of converting the binary form into the decimal form; if the compression mode is marked as a first-order difference mark, decompressing the data to be decompressed included in the sub data segment to be decompressed according to a decompression mode corresponding to the first-order difference compression; and if the compression mode mark is a second-order difference mark, decompressing the data to be decompressed included in the sub-data segment to be decompressed according to a decompression mode corresponding to the second-order difference compression.
As a possible implementation, the data processing apparatus further includes:
the migration unit 909 is configured to perform migration on the data obtained after decompression according to the migration amount included in the compression parameter, so as to obtain a corresponding to-be-processed data segment before the to-be-decompressed data segment is compressed.
The processing unit 904 is further configured to recombine the to-be-processed data segments obtained by decompressing all the compressed data segments included in the compressed data array according to the time sequence, so as to obtain a to-be-processed data set.
According to the embodiment provided by the present invention, the obtaining unit 901 obtains the value range of the to-be-processed data included in the to-be-processed data set, and the determining unit 902 is used to determine the compression parameter for the to-be-processed data set according to the value range of the to-be-processed data included in the to-be-processed data set. The dividing unit 903 offsets the to-be-processed data included in the to-be-processed data set according to the offset, and divides the offset to-be-processed data into a plurality of to-be-processed data segments according to the maximum compression length. The determining unit 902 determines a compression mode according to the current position of the pointing cursor, specifically including second-order differential compression and first-order differential compression; and compressing the data to be processed according to the determined compression mode, and adding the compressed data into the compressed data array in sequence. It can be seen that the implementation manner can achieve higher compression efficiency on the data to be processed with smaller calculation amount.
Referring to fig. 10, fig. 10 is a schematic block diagram of another data processing apparatus according to an embodiment of the present invention, where the apparatus may include: one or more processors 1001, one or more communication interfaces 1002 and a memory 1003, the processors 1001, the communication interfaces 1002 and the memory 1003 being interconnected via a bus 1004, wherein the memory 1003 is configured to store a computer program comprising program instructions, the processors 1001 being configured to invoke the program instructions. In an embodiment of the present invention, one or more instructions stored in a computer storage medium are loaded and executed by processor 1001 to implement the corresponding steps of the method flows shown in fig. 1 or 3; in a particular implementation, one or more instructions in the computer storage medium are loaded by the processor 1001 and perform the following steps:
and acquiring compression parameters aiming at the data set to be processed, wherein the compression parameters comprise an offset, a maximum compression length and a control frame mark.
The data to be processed included in the data set to be processed is shifted according to the shift amount, and the shifted data to be processed is divided into a plurality of data segments to be processed according to the maximum compression length;
and respectively compressing the plurality of data segments to be processed, and respectively adding the compressed data into the compressed data arrays according to the time sequence by using the control frame marks corresponding to the compression mode adopted in the compression process.
As a possible implementation manner, the specific implementation manner of the processor 1001 to obtain the compression parameters for the data set to be processed may be: acquiring a value range of to-be-processed data included in a to-be-processed data set; according to the value range of the data to be processed included in the data set to be processed, determining the offset, the frame length, the first-order frame length, the second-order frame length and a control frame mark aiming at the data set to be processed, wherein the control frame mark comprises a starting frame mark, a compression mode mark and an ending frame mark.
As a possible implementation manner, a specific implementation manner of the processor 1001 respectively compressing the multiple pieces of data to be processed may be: determining the current position of an indicating cursor in a target data segment to be processed aiming at the target data segment to be processed in a plurality of data segments to be processed, wherein the indicating cursor is used for indicating the next data segment to be processed, and the target data segment to be processed is any one of the data segments to be processed; if the current position of the indication cursor does not exceed the position of the last data to be processed in the target data segment to be processed, determining a compression mode according to the compression condition met by the current position of the indication cursor; and compressing the target data segment to be processed according to the determined compression mode.
As a possible implementation manner, the processor 1001 is further configured to: and if the current indication of the indication cursor is the data to be processed which is arranged at the forefront in the target data segment to be processed, adding a starting frame mark at the first position of the target compressed data segment corresponding to the target data segment to be processed in the compressed data array.
As a possible implementation manner, the compression mode flag further includes a second order difference flag, the compression mode includes second order difference compression, and the specific implementation manner of the processor 1001 compressing the target data segment to be processed according to the determined compression mode may be: judging whether the current position of the indicating cursor meets a compression condition corresponding to the second-order differential compression; if yes, performing second-order differential compression on the data indicating the current position of the cursor and the frame length/second-order frame length behind the current position; judging whether a compression mode mark arranged at the last in the target compression data segment is a second-order difference mark or not; if the second-order difference mark exists, adding the data after the second-order difference compression at the next position of the target compressed data segment; if the target compressed data segment is not the second-order differential mark, adding the second-order differential mark at the next position of the target compressed data segment, and adding the data after the second-order differential compression at the next position of the second-order differential mark; the cursor will be instructed to move back by the frame length/second order frame length positions.
As a possible implementation manner, the compression mode flag further includes a first order difference flag, and the compression mode further includes a first order difference compression, and if the current position of the cursor does not satisfy the compression condition corresponding to the second order difference compression, the processor 1001 is further configured to: judging whether the current position of the indicating cursor meets a compression condition corresponding to first-order differential compression; if yes, performing first-order differential compression on the position of the indication cursor and the frame length/first-order frame length data behind the position; judging whether a compression mode mark arranged at the last in the target compression data section is a first-order difference mark or not; if the first-order difference mark exists, adding the data after the first-order difference compression at the next position of the target compressed data section; if the first-order difference mark is not the first-order difference mark, adding the first-order difference mark at the next position of the target compressed data section, and adding the data after the first-order difference compression at the next position of the first-order difference mark; the cursor will be indicated to move back by the frame length/first order frame length position.
As a possible implementation manner, if the position where the pointing cursor is currently located does not satisfy the compression condition corresponding to the first-order difference compression, the processor 1001 is further configured to: converting the data to be processed currently indicated by the indication cursor into binary data; judging whether a compression mode mark arranged at the last in the target compression data segment is a key frame mark or not; if the key frame mark exists, adding data in a binary form at the next position of the target compressed data segment; if the target compressed data segment is not the key frame mark, adding the key frame mark at the next position of the target compressed data segment, and adding the binary form data at the next position of the key frame mark; the pointing cursor is moved back by one position.
As one possible embodiment, the compression condition includes a data length condition, a compression feasibility condition, and a collision avoidance condition; the data length conditions of the second order differential compression are as follows: the front of the position of the indicating cursor is provided with at least two compressed data, and the position and the back of the indicating cursor are provided with at least frame length/second-order frame length data; the compression feasibility conditions of the second-order differential compression are as follows: indicating the position of the cursor and the second-order differential value of the frame length/second-order frame length data behind the cursor are smaller than the maximum value of the second-order frame, wherein the maximum value of the second-order frame is the maximum value represented by binary number of the second-order frame length; the collision avoidance conditions for the second order differential compression are: the data obtained after the second-order differential compression is different from the control frame mark; the data length condition of the first order difference compression is as follows: indicating at least one piece of compressed data in front of the position of the cursor, and indicating at least one piece of frame length/first-order frame length data behind the position of the cursor; the compression feasibility conditions for the first order differential compression are: indicating the position of the cursor and the first-order difference value of the frame length/first-order frame length data behind the cursor are all smaller than the maximum value of the first-order frame, and the maximum value of the first-order frame is the maximum value represented by binary number of the first-order frame length; the collision avoidance conditions for first order differential compression are: the data obtained after the first order difference compression is different from the control frame mark.
As a possible implementation manner, the processor 1001 is further configured to: determining a data segment to be decompressed according to the compression parameters, wherein the data segment to be decompressed is any compressed data segment in which a first frame in a compressed data array is a starting frame mark and a last frame is an ending frame mark; dividing the data segment to be decompressed into a plurality of sub data segments to be decompressed according to the compression mode marks included in the data segment to be decompressed, wherein each sub data segment to be decompressed comprises the respective compression mode mark and the data to be decompressed; and respectively decompressing the plurality of sub data segments to be decompressed according to the respective compression mode marks.
As a possible implementation manner, the specific implementation manner of the processor 1001 respectively decompressing the multiple sub data segments to be decompressed according to the respective compression mode flags may be: if the compression mode is marked as a key frame mark, decompressing the data to be decompressed included in the sub-data segment to be decompressed by utilizing a mode of converting the binary form into the decimal form; if the compression mode is marked as a first-order difference mark, decompressing the data to be decompressed included in the sub data segment to be decompressed according to a decompression mode corresponding to the first-order difference compression; if the compression mode mark is a second-order difference mark, decompressing the data to be decompressed included in the sub data segment to be decompressed according to a decompression mode corresponding to the second-order difference compression; and offsetting the data obtained after decompression according to the offset included by the compression parameters to obtain the corresponding data segment to be processed before the data segment to be decompressed is compressed.
As a possible implementation manner, the processor 1001 is further configured to: and recombining the to-be-processed data segments obtained by decompressing all the compressed data segments included in the compressed data array according to the time sequence to obtain a to-be-processed data set.
In another embodiment of the present invention, a computer-readable storage medium is provided, which stores a computer program, the computer program comprising program instructions, which when executed by a processor, implement the implementation of the data processing method described in fig. 1 or fig. 3 provided by the embodiment of the present invention.
The computer readable storage medium may be an internal storage unit in the data processing platform according to any of the foregoing embodiments, or a hard disk or a memory of a server. The computer readable storage medium may also be an external storage device of the server, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the server.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.

Claims (15)

  1. A method of data processing, the method comprising:
    acquiring compression parameters aiming at a data set to be processed, wherein the compression parameters comprise an offset, a maximum compression length and a control frame mark;
    shifting the data to be processed included in the data set to be processed according to the offset, and dividing the shifted data to be processed into a plurality of data segments to be processed according to the maximum compression length;
    and respectively compressing the plurality of data segments to be processed, and adding the compressed data into the compressed data arrays according to the time sequence by using the control frame marks corresponding to the compression mode adopted in the compression process.
  2. The method of claim 1, wherein the compression parameters further comprise a frame length, a first order frame length, and a second order frame length, and wherein obtaining compression parameters for a set of data to be processed comprises:
    acquiring a value range of to-be-processed data included in a to-be-processed data set;
    and determining the offset, the frame length, the first-order frame length, the second-order frame length and the control frame mark aiming at the data set to be processed according to the value range of the data to be processed included in the data set to be processed, wherein the control frame mark comprises a starting frame mark, a compression mode mark and an ending frame mark.
  3. The method of claim 2, wherein the start frame marker is used to mark the beginning of a compressed data segment in the compressed data array, the start frame marker being located at a first position of the compressed data segment;
    the end frame marker is used for marking the end of one compressed data segment in the compressed data array, and the end frame marker is located at the last position of the compressed data segment.
  4. The method of claim 2 or 3, wherein the compressing the plurality of to-be-processed data segments respectively comprises:
    determining the current position of an indicating cursor in a target data segment to be processed aiming at the target data segment to be processed in the plurality of data segments to be processed, wherein the indicating cursor is used for indicating the next data to be processed, and the target data segment to be processed is any one of the plurality of data segments to be processed;
    if the current position of the indication cursor does not exceed the position of the last to-be-processed data arranged in the target to-be-processed data segment, determining a compression mode according to a compression condition met by the current position of the indication cursor;
    and compressing the target data segment to be processed according to the determined compression mode.
  5. The method of claim 4, wherein the method further comprises:
    and if the current indication of the indication cursor is the data to be processed which is arranged at the forefront in the target data segment to be processed, adding the start frame mark at the first position of the target compressed data segment corresponding to the target data segment to be processed in the compressed data array.
  6. The method according to any of claims 4 or 5, wherein the compression mode flag comprises a second order differential flag, the compression mode comprises a second order differential compression, and the compressing the target data segment to be processed according to the determined compression mode comprises:
    judging whether the current position of the indicating cursor meets a compression condition corresponding to the second-order differential compression;
    if yes, performing second-order differential compression on the current position of the indicating cursor and the frame length/second-order frame length data behind the current position;
    judging whether the compression mode mark arranged at the last in the target compression data segment is the second-order difference mark or not;
    if the second-order difference mark exists, adding the data after the second-order difference compression at the next position of the target compressed data segment;
    if the second-order difference mark is not the second-order difference mark, adding the second-order difference mark at the next position of the target compressed data segment, and adding the data after the second-order difference compression at the next position of the second-order difference mark;
    and moving the indication cursor backwards by the frame length/the second-order frame length.
  7. The method of claim 6, wherein the compression mode flag further comprises a first order difference flag, the compression mode further comprises a first order difference compression, and if the current position of the pointing cursor does not satisfy the compression condition corresponding to the second order difference compression, the method further comprises:
    judging whether the current position of the indicating cursor meets a compression condition corresponding to the first-order differential compression;
    if yes, performing first-order differential compression on the current position of the indicating cursor and the frame length/first-order frame length data behind the current position;
    judging whether the compression mode mark arranged at the last in the target compression data section is the first-order difference mark or not;
    if the first-order difference mark exists, adding the data after the first-order difference compression at the next position of the target compressed data section;
    if the first-order difference mark is not the first-order difference mark, adding the first-order difference mark at the next position of the target compressed data section, and adding the first-order difference compressed data at the next position of the first-order difference mark;
    moving the indicator cursor backward the frame length/the first order frame length positions.
  8. The method of claim 7, wherein the compression mode flag further comprises a key frame flag, and if the current position of the pointing cursor does not satisfy the compression condition corresponding to the first order difference compression, the method further comprises:
    converting the data to be processed currently indicated by the indication cursor into binary form data;
    judging whether the compression mode mark arranged at the last in the target compression data segment is the key frame mark or not;
    if the key frame mark exists, adding the binary data at the next position of the target compressed data segment;
    if the target compressed data segment is not the key frame mark, adding the key frame mark at the next position of the target compressed data segment, and adding the binary form data at the next position of the key frame mark;
    and moving the indication cursor backwards by one position.
  9. The method of claims 4-8, wherein the compression conditions include a data length condition, a compression feasibility condition, and a collision avoidance condition;
    the data length condition of the second-order differential compression is as follows: the front of the current position of the indicating cursor is provided with at least two pieces of compressed data, and the current position and the back of the indicating cursor are provided with at least the frame length/the second-order frame length data; the compression feasibility condition of the second-order differential compression is as follows: the current position of the indicating cursor and the second-order differential value of the frame length/the second-order frame length data behind the indicating cursor are both smaller than the maximum value of a second-order frame, and the maximum value of the second-order frame is the maximum value represented by binary number of the length of the second-order frame; the collision avoidance condition of the second-order differential compression is as follows: the data obtained after the second-order differential compression is different from the control frame mark;
    the data length condition of the first-order difference compression is as follows: the front of the current position of the indicating cursor is provided with at least one piece of compressed data, and the current position and the back of the indicating cursor are provided with at least one piece of data of the frame length/the first-order frame length; the compression feasibility condition of the first-order differential compression is as follows: the current position of the indicating cursor and the first-order difference value of the frame length/first-order frame length data behind the indicating cursor are all smaller than the maximum value of a first-order frame, and the maximum value of the first-order frame is the maximum value represented by binary number of the length of the first-order frame; the collision avoidance condition of the first order differential compression is as follows: and the data obtained after the first-order difference compression is different from the control frame mark.
  10. The method of claim 1, wherein the control frame marker comprises a start frame marker, a compressed mode marker, and an end frame marker, the compressed data array comprising at least one compressed data segment, the method further comprising:
    determining a data segment to be decompressed according to the compression parameters, wherein the data segment to be decompressed is any compressed data segment in which a first frame in the compressed data array is a starting frame mark and a last frame is an ending frame mark;
    dividing the data segment to be decompressed into a plurality of sub data segments to be decompressed according to the compression mode marks included in the data segment to be decompressed, wherein each sub data segment to be decompressed comprises the respective compression mode mark and the data to be decompressed;
    and respectively decompressing the plurality of sub data segments to be decompressed according to the respective compression mode marks.
  11. The method of claim 10, wherein the compression mode flag comprises a key frame flag, a first order difference flag, and a second order difference flag, and wherein decompressing the plurality of sub-data segments to be decompressed according to the respective compression mode flags respectively comprises:
    if the compression mode mark is the key frame mark, decompressing the data to be decompressed included in the sub-data segment to be decompressed by utilizing a mode of converting a binary form into a decimal form;
    if the compression mode mark is the first-order difference mark, decompressing the data to be decompressed included in the sub data segment to be decompressed according to a decompression mode corresponding to the first-order difference compression;
    if the compression mode mark is the second-order differential mark, decompressing the data to be decompressed included in the sub-data segment to be decompressed according to a decompression mode corresponding to the second-order differential compression;
    and offsetting the data obtained after decompression according to the offset included by the compression parameters to obtain the corresponding data segment to be processed before the data segment to be decompressed is compressed.
  12. The method of claim 11, wherein the method further comprises:
    and recombining the to-be-processed data segments obtained by decompressing all the compressed data segments included in the compressed data array according to the time sequence to obtain the to-be-processed data set.
  13. A data processing apparatus comprising means for performing the method of any one of claims 1 to 12.
  14. A data processing apparatus comprising a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory being interconnected, wherein the memory is adapted to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the method of any of claims 1 to 12.
  15. A computer-readable storage medium, characterized in that the computer storage medium stores a computer program comprising program instructions that, when executed by a processor, cause the processor to perform the method according to any of claims 1-12.
CN201880086748.4A 2018-04-03 2018-04-03 Data processing method and device Pending CN111602111A (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/081757 WO2019191904A1 (en) 2018-04-03 2018-04-03 Data processing method and device

Publications (1)

Publication Number Publication Date
CN111602111A true CN111602111A (en) 2020-08-28

Family

ID=68099742

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201880086748.4A Pending CN111602111A (en) 2018-04-03 2018-04-03 Data processing method and device

Country Status (2)

Country Link
CN (1) CN111602111A (en)
WO (1) WO2019191904A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1317882A (en) * 2001-03-22 2001-10-17 谭伟祥 Method for compressing and decompressing data in database
CN101355364A (en) * 2008-09-08 2009-01-28 北大方正集团有限公司 Method and apparatus for compressing and decompressing file
CN105009584A (en) * 2013-04-02 2015-10-28 明达半导体股份有限公司 Method and apparatus for processing video

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160345260A1 (en) * 2005-05-30 2016-11-24 Invent.ly LLC Self Powered Device and Methods Conserving Energy in Communication
CN103098471B (en) * 2010-09-14 2016-07-06 三星电子株式会社 Equipment and method for multilayer picture optical coding/decoding
CN103309621B (en) * 2012-12-12 2016-03-16 珠海金山网络游戏科技有限公司 Read the method for data in sectional compression bag
CN107295036B (en) * 2016-03-31 2021-02-23 华为技术有限公司 Data sending method and data merging equipment

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1317882A (en) * 2001-03-22 2001-10-17 谭伟祥 Method for compressing and decompressing data in database
CN101355364A (en) * 2008-09-08 2009-01-28 北大方正集团有限公司 Method and apparatus for compressing and decompressing file
CN105009584A (en) * 2013-04-02 2015-10-28 明达半导体股份有限公司 Method and apparatus for processing video

Also Published As

Publication number Publication date
WO2019191904A1 (en) 2019-10-10

Similar Documents

Publication Publication Date Title
US9729169B2 (en) Methods and devices for source-coding and decoding of data involving symbol compression
CN105634499B (en) Data conversion method based on new short floating point type data
CN116506073B (en) Industrial computer platform data rapid transmission method and system
EP3108583B1 (en) Devices and methods of source-encoding and decoding of data
CN111159492A (en) Track point retrieval method and device, storage medium and terminal
JP2013251895A (en) Method and device for compressing and decompressing signal data (compression and decompression of signal data)
CN110825323B (en) Storage and reading method of floating point number data and computer readable storage medium
CN109962711B (en) Data compression method, electronic equipment and storage medium
CN115208414B (en) Data compression method, data compression device, computer device and storage medium
CN113765620B (en) Block compression encoding method, block compression encoding device, computer equipment and readable storage medium
CN113630125A (en) Data compression method, data encoding method, data decompression method, data encoding device, data decompression device, electronic equipment and storage medium
CN111491169B (en) Digital image compression method, device, equipment and medium
CN111061722B (en) Data compression and data decompression method, device and equipment
CN110442557B (en) Data compression and decompression method, electronic device and computer readable storage medium
CN111602111A (en) Data processing method and device
CN112332854A (en) Hardware implementation method and device of Huffman coding and storage medium
US20190199372A1 (en) Information processing apparatus and information processing method
US10340946B2 (en) Encoders, decoders, and methods
CN112054805B (en) Model data compression method, system and related equipment
CN114692077A (en) Matrix calculation device, method, system, circuit, chip and equipment
CN111417920A (en) Data processing method and device
CN111259177B (en) Black-white binary signature picture storage method and system
CN112783056B (en) Data programming method, device and equipment of ECU and storage medium
CN111697973B (en) Compression method and compression system
CN114547030B (en) Multi-stage time sequence data compression method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: 518172 43 Universiade Software Town, 8288 Longgang Avenue, Henggang Street, Longgang District, Shenzhen City, Guangdong Province

Applicant after: Shenzhen Ruoyu Technology Co.,Ltd.

Address before: 518172 43 Universiade Software Town, 8288 Longgang Avenue, Henggang Street, Longgang District, Shenzhen City, Guangdong Province

Applicant before: SHENZHEN ROYOLE TECHNOLOGIES Co.,Ltd.

CB02 Change of applicant information
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20200828

WD01 Invention patent application deemed withdrawn after publication